Systems and methods for adapting content items to endpoint media devices

ABSTRACT

Systems, methods, and machine-readable media for adapting content items to device operations of an endpoint media device are disclosed. A first content composite may be created by one or more processing devices for delivery in a packet stream from a content provider system via one or more networks, where the first content composite may include an adaptable content item corresponding to a set of one or more audio and/or video packets corresponding to audio and/or video content. The first content composite may be transmitted, where, consequent to delivery of the first content composite to a first endpoint media device or a second endpoint media device, the first endpoint media device or the second endpoint media device may perform one operation relating to a second content item of the first content composite, where the adaptable content item may be modified based on the second content item.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.16/655,104, filed Oct. 16, 2019, now issued as U.S. Pat. No. 10,880,351,the entire contents of which are hereby incorporated by reference forall purposes.

BACKGROUND

This disclosure generally relates to content delivery, and moreparticularly to systems and methods for adapting content items toendpoint media devices.

The evolution of wireless networks and content distribution platformsmay present a number of challenges and problems for content deliverywithin packet streams per Internet Protocol (IP), the ATSC (AdvancedTelevision Systems Committee) 3.0 standard, or a like protocol. Suchmodes of content delivery may not be linear, in contrast to traditionaltelevision delivery, for example. And, still, there are needs for deeperintegration with endpoint devices in a broadcast environment. Contentviewers are in need of better viewer experiences and more tailoredservice offerings.

Thus, there is a need for systems and methods that address suchproblems. This and other needs are addressed by the present disclosure.

BRIEF SUMMARY

Certain embodiments of the present disclosure relate generally tocontent delivery, and more particularly to systems and methods foradapting content items to endpoint media devices.

In one aspect, a method for adapting content items to device operationsof an endpoint media device is disclosed. The method may include one ora combination of the following. A first content composite may be createdby one or more processing devices for delivery in a packet stream from acontent provider system via one or more networks, where the firstcontent composite may include an adaptable content item corresponding toa set of one or more audio and/or video packets corresponding to audioand/or video content. Transmission of the first content composite may becaused, where, consequent to delivery of the first content composite toa first endpoint media device or a second endpoint media device, thefirst endpoint media device or the second endpoint media device mayperform at least one operation relating to a second content item of thefirst content composite represented by one or more images, where theadaptable content item may be modified based at least in part on thesecond content item. At least one communication received from the firstendpoint media device, the second endpoint media device, or a remotesystem that is remote from the one or more processing devices, the firstendpoint media device, and the second endpoint media device, may beprocessed. The at least one communication may be indicative of the atleast one operation relating to the second content item of the firstcontent composite. Based at least in part on the processing the at leastone communication, a set of observation data corresponding toindications of detected operations associated with the first endpointmedia device and/or the second endpoint media device and mapped to a setof one or more content items may be aggregated. The set of one or morecontent items may include the second content item. Based at least inpart on the set of observation data, a second content composite may beadapted so that the second content composite is modified based at leastin part on a third content item. The second content composite may beprovided for presentation facilitated by one or both of the firstendpoint media device and the second endpoint media device.

In another aspect, a system to adapt content items to device operationsof an endpoint media device is disclosed. The system may include one ormore processing devices and memory communicatively coupled with andreadable by the one or more processing devices and having stored thereinprocessor-readable instructions which, when executed by the one or moreprocessing devices, cause the one or more processing devices to performone or a combination of the following. A first content composite may becreated by one or more processing devices for delivery in a packetstream from a content provider system via one or more networks, wherethe first content composite may include an adaptable content itemcorresponding to a set of one or more audio and/or video packetscorresponding to audio and/or video content. Transmission of the firstcontent composite may be caused, where, consequent to delivery of thefirst content composite to a first endpoint media device or a secondendpoint media device, the first endpoint media device or the secondendpoint media device may perform at least one operation relating to asecond content item of the first content composite represented by one ormore images, where the adaptable content item may be modified based atleast in part on the second content item. At least one communicationreceived from the first endpoint media device, the second endpoint mediadevice, or a remote system that is remote from the one or moreprocessing devices, the first endpoint media device, and the secondendpoint media device, may be processed. The at least one communicationmay be indicative of the at least one operation relating to the secondcontent item of the first content composite. Based at least in part onthe processing the at least one communication, a set of observation datacorresponding to indications of detected operations associated with thefirst endpoint media device and/or the second endpoint media device andmapped to a set of one or more content items may be aggregated. The setof one or more content items may include the second content item. Basedat least in part on the set of observation data, a second contentcomposite may be adapted so that the second content composite ismodified based at least in part on a third content item. The secondcontent composite may be provided for presentation facilitated by one orboth of the first endpoint media device and the second endpoint mediadevice.

In yet another aspect, one or more non-transitory, machine-readablemedia are disclosed. The one or more non-transitory, machine-readablemedia may have machine-readable instructions thereon which, whenexecuted by one or more processing devices, cause the one or moreprocessing devices to perform one or a combination of the following. Afirst content composite may be created by one or more processing devicesfor delivery in a packet stream from a content provider system via oneor more networks, where the first content composite may include anadaptable content item corresponding to a set of one or more audioand/or video packets corresponding to audio and/or video content.Transmission of the first content composite may be caused, where,consequent to delivery of the first content composite to a firstendpoint media device or a second endpoint media device, the firstendpoint media device or the second endpoint media device may perform atleast one operation relating to a second content item of the firstcontent composite represented by one or more images, where the adaptablecontent item may be modified based at least in part on the secondcontent item. At least one communication received from the firstendpoint media device, the second endpoint media device, or a remotesystem that is remote from the one or more processing devices, the firstendpoint media device, and the second endpoint media device, may beprocessed. The at least one communication may be indicative of the atleast one operation relating to the second content item of the firstcontent composite. Based at least in part on the processing the at leastone communication, a set of observation data corresponding toindications of detected operations associated with the first endpointmedia device and/or the second endpoint media device and mapped to a setof one or more content items may be aggregated. The set of one or morecontent items may include the second content item. Based at least inpart on the set of observation data, a second content composite may beadapted so that the second content composite is modified based at leastin part on a third content item. The second content composite may beprovided for presentation facilitated by one or both of the firstendpoint media device and the second endpoint media device.

In various embodiments, the content composite may be created least inpart by configuring a containerization object with a container flag thatindicates the adaptable content item and containerization specificationsto facilitate performance of a set of one or more operations by thefirst endpoint media device and/or the second endpoint media device withrespect to the adaptable content item consequent to the delivery of thecontent composite. In various embodiments, the at least one operationrelating to the second content item may correspond to a selection of auser-selectable interface element mapped to the second content item. Invarious embodiments, consequent to the selection of a user-selectableinterface element mapped to the second content item, at least a portionof the second content item may be transferred to one or both of thefirst endpoint media device and the second endpoint media device.

In various embodiments, the at least one operation relating to thesecond content item may correspond to an image capture of at least aportion of an image facilitated by the second content item. In variousembodiments, based at least in part on the aggregated set of observationdata, one or more metrics of media device interactions with a set of oneor more content composites may be learned by the one or more processingdevices, and the one or more metrics may be mapped to the first endpointmedia device and/or the second endpoint media device. The adapting thesecond content composite may be based at least in part on the learnedone or more metrics of media device interactions. In variousembodiments, the one or more metrics of media device may correspond toindicia of one or more times of: the first endpoint media device and/orthe second endpoint media device having output the second content itemfor display, and/or a selection of a user-selectable interface elementmapped to the second content item

Further areas of applicability of the present disclosure will becomeapparent from the detailed description provided hereinafter. It shouldbe understood that the detailed description and specific examples, whileindicating various embodiments, are intended for purposes ofillustration only and are not intended to necessarily limit the scope ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of variousembodiments may be realized by reference to the following figures. Inthe appended figures, similar components or features may have the samereference label. Further, various components of the same type may bedistinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

FIG. 1 illustrates a content distribution system, in accordance withdisclosed embodiments of the present disclosure.

FIG. 2 further illustrates a content distribution system, in accordancewith disclosed embodiments of the present disclosure.

FIG. 3 illustrates a diagram of an adaptive content composite system, inaccordance with disclosed embodiments of the present disclosure.

FIG. 4 illustrates a composite build engine, in accordance withdisclosed embodiments of the present disclosure.

FIG. 5 illustrates certain aspects of subsystem data flow, in accordancewith disclosed embodiments of the present disclosure.

FIG. 6 illustrates a receiver that makes use of, interacts with, and/orat least partially includes the content composite subsystem, inaccordance with disclosed embodiments of the present disclosure.

FIG. 7 illustrates a system including one non-limiting example of acomputing device configured to make use of, interact with, and/or atleast partially include the content composite subsystem, in accordancewith disclosed embodiments of the present disclosure.

FIG. 8 illustrates a computer system, in accordance with disclosedembodiments of the present disclosure.

DETAILED DESCRIPTION

The ensuing description provides preferred exemplary embodiment(s) only,and is not intended to limit the scope, applicability or configurationof the disclosure. Rather, the ensuing description of the preferredexemplary embodiment(s) will provide those skilled in the art with anenabling description for implementing a preferred exemplary embodimentof the disclosure. It should be understood that various changes may bemade in the function and arrangement of elements without departing fromthe spirit and scope of the disclosure as set forth in the appendedclaims.

Specific details are given in the following description to provide athorough understanding of the embodiments. However, it will beunderstood by one of ordinary skill in the art that the embodimentsmaybe practiced without these specific details. For example, circuitsmay be shown in block diagrams in order not to obscure the embodimentsin unnecessary detail. In other instances, well-known circuits,processes, algorithms, structures, and techniques may be shown withoutunnecessary detail in order to avoid obscuring the embodiments.

Also, it is noted that the embodiments may be described as a processwhich is depicted as a flowchart, a flow diagram, a data flow diagram, astructure diagram, or a block diagram. Although a flowchart may describethe operations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be rearranged. A process is terminated when itsoperations are completed, but could have additional steps not includedin the figure. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination corresponds to a return of the functionto the calling function or the main function.

Various embodiments will now be discussed in greater detail withreference to the accompanying figures, beginning with FIG. 1 .

FIG. 1 illustrates a content distribution system 100, in accordance withdisclosed embodiments of the present disclosure. For brevity, system 100is depicted in a simplified and conceptual form, and may generallyinclude more or fewer systems, devices, networks, and/or othercomponents as desired. Further, the number and types of features orelements incorporated within the system 100 may or may not beimplementation-specific, and at least some of the aspects of the system100 may be similar to a cable television distribution system, an IPTV(Internet Protocol Television) content distribution system, and/oranother type of media or content distribution system.

In general, the system 100 may include a plurality of networks 120 thatcan be used for bi-directional communication paths for data transferbetween components of system 100. Disclosed embodiments may transmit andreceive data, including video content, via the networks 120 using anysuitable protocol(s), as is disclosed in co-pending U.S. patentapplication Ser. No. 16/383,300, filed Apr. 12, 2019, the disclosure ofwhich application is incorporated by reference herein in its entiretyfor all purposes, as if fully set forth herein. The networks 120 may beor include one or more next-generation networks (e.g., 5G wirelessnetworks and beyond). Further, the plurality of networks 120 maycorrespond to a hybrid network architecture with any number ofterrestrial and/or non-terrestrial networks and/or network features, forexample, cable, satellite, wireless/cellular, or Internet systems, orthe like, utilizing various transport technologies and/or protocols,such as radio frequency (RF), optical, satellite, coaxial cable,Ethernet, cellular, twisted pair, other wired and wireless technologies,and the like. In various instances, the networks 120 may be implementedwith, without limitation, satellite communication with a plurality oforbiting (e.g., geosynchronous) satellites, a variety of wirelessnetwork technologies such as 5G, 4G, LTE (Long-Term Evolution), 3G, GSM(Global System for Mobile Communications), another type of wirelessnetwork (e.g., a network operating under Bluetooth®, any of theInstitute of Electrical and Electronics (IEEE) 802.11 suite ofprotocols, and/or any other wireless protocol), a wireless local areanetwork (WLAN), a HAN (Home Area Network) network, another type ofcellular network, the Internet, a wide area network (WAN), a local areanetwork (LAN) such as one based on Ethernet, Token-Ring and/or the like,such as through etc., a gateway, and/or any other appropriatearchitecture or system that facilitates the wireless and/or hardwiredpacket-based communications of signals, data, and/or message inaccordance with embodiments disclosed herein. In various embodiments,the networks 120 and its various components may be implemented usinghardware, software, and communications media such wires, optical fibers,microwaves, radio waves, and other electromagnetic and/or opticalcarriers; and/or any combination of the foregoing and/or the like. Insome embodiments, the networks 120 may include a telephone network thatmay be circuit switched, package switched, or partially circuit switchedand partially package switched. For example, the telephone network maypartially use the Internet to carry phone calls (e.g., through VoIP). Invarious instances, the networks 120 may transmit data using any suitablecommunication protocol(s), such as TCP/IP (Transmission ControlProtocol/Internet protocol), SNA (systems network architecture), IPX(Internet packet exchange), UDP, AppleTalk, and/or the like.

The system 100 may include one or more content provider systems 102, oneor more other data source systems 103, and media devices includingadaptive routers 110 and various endpoint devices 116. Many embodimentsmay include a large number of content provider systems 102, data sourcesystems 103, and/or such media devices. The one or more content providersystems 102 may include one or more television service provider systems.The content provider systems 102 may distribute broadcast video contentto the endpoint devices 116 via one or more networks of the networks 120and the adaptive routers 110. For example, a content provider system 102may be configured to stream, via the networks 120, television channels,on-demand programing, movies, other shows, television programs orportions of television programs following and/or during a live broadcastof the television programs, an content and commercials, programminginformation (e.g., table data, electronic programming guide (EPG)content, etc.), and/or other services to endpoint devices 116 viasatellite, 5G, 4G, and/or LTE wireless communication protocols andnetwork components, in accordance with embodiments disclosed herein.

A content provider system 102 with satellite transmitter equipment,satellite uplink, and/or other network routing equipment that may, forexample, be operated by a television service provider. The contentprovider system 102 may receive feeds of one or more television channelsfrom various sources. Such television channels may include multipletelevision channels that contain the same content (but may be indifferent formats, such as high-definition and standard-definition). Todistribute such video content to endpoint devices 116, feeds of thevideo content may be relayed to the adaptive routers 110 and theendpoint devices 116 via one or more satellites in the form oftransponder streams or transmitted to the adaptive routers 110 and theendpoint devices 116.

The data source systems 103 may correspond to any suitable sources ofdata such as one or more computer systems, databases, websites, portals,any repositories of data in any suitable form, server systems, otherendpoint devices like endpoint devices 116 but functioning as datasources, and/or the like. In some instances, the data source systems 103may include one or more mobile computing device locator services thatprovide information regarding the location of one or more of theendpoint devices 116 and/or the adaptive routers 110. In variousinstances, the data source systems 103 may provide various detailsrelating to IP addresses, cellular tower identification and locationdata, mobile device triangulation data, LAN identification data, Wi-Fiidentification data, access point identification and location data,and/or the like data that facilitates location of one or more of theendpoint devices 116 and/or the adaptive routers 110.

In various embodiments, the data from one or more of the data sourcesystems 103 may be retrieved and/or received by a television serviceprovider system 102 via one or more data acquisition interfaces throughnetwork(s) 120 and/or through any other suitable means of transferringdata. In various embodiments, the television service provider system 102the data source systems 103 could use any suitable means for directcommunication. In various embodiments, the television service providersystem 102 may actively gather and/or pull from one or more of the datasource systems 103. Additionally or alternatively, the televisionservice provider system 102 may wait for updates from one or more of thedata source systems 103. The data collected (location data, IP address,etc.) may be curated so that only the data necessary for the transactionis collected. The one or more data acquisition interfaces may includeone or more application programming interfaces (APIs) that defineprotocols and routines for interfacing with the data source systems 103.The APIs may specify application programming interface (API) callsto/from data source systems 103. In some embodiments, the APIs mayinclude a plug-in to integrate with an application of a data sourcesystems 103. The data acquisition interfaces, in some embodiments, coulduse a number of API translation profiles configured to allow interfacewith the one or more additional applications of the data sources toaccess data (e.g., a database or other data store) of the data sourcesystems 103. The API translation profiles may translate the protocolsand routines of the data source systems 103 to integrate at leasttemporarily with the system and allow communication with the system byway of API calls.

FIG. 2 illustrates a content distribution system 200, in accordance withdisclosed embodiments of the present disclosure. In various embodiments,the system 200 may correspond to an example system that may encompassall or part of the content distribution system 100 and that illustratescertain features of disclosed embodiments. Again, for brevity, system200 is depicted in a simplified and conceptual form, and may generallyinclude more or fewer systems, devices, networks, and/or othercomponents as desired. Further, the number and types of features orelements incorporated within the system 200 may or may not beimplementation-specific, and at least some of the aspects of the system200 may be similar to a cable television distribution system, an IPTV(Internet Protocol Television) content distribution system, and/or anyother type of media or content distribution system.

The system 200 may include content provider system 102-1, satelliteuplink 104, a plurality of orbiting (e.g., geosynchronous) satellites106, satellite receiver 108, one or more computing devices 116 a-d, oneor more television receivers 116 e, one or more content sources 112(e.g., online content sources), and service provider systems 103-1. Invarious embodiments, one or a combination of the one or more computingdevices 116 a-d, one or more television receivers 116 e, and/or one ormore display devices 160 may correspond to endpoint media devices 116.

The content provider system 102-1 and satellite transmitter equipment(which may include the satellite uplink 104) may be operated by atelevision service provider. A television service provider maydistribute television channels, on-demand programing, programminginformation, and/or other services to users via satellite. The contentprovider system 102-1 may receive feeds of one or more televisionchannels from various sources. Such television channels may includemultiple television channels that contain the same content (but may bein different formats, such as high-definition and standard-definition).To distribute such television channels to users, feeds of the televisionchannels may be relayed to user equipment via one or more satellites inthe form of transponder streams. Satellite transmitter equipment may beused to transmit a feed of one or more television channels from thecontent provider system 102-1 to one or more satellites 106. While asingle content provider system 102-1 and satellite uplink 104 areillustrated as part of the television distribution system 200, it shouldbe understood that multiple instances of transmitter equipment may beused, possibly scattered geographically to communicate with satellites106. Such multiple instances of satellite transmitting equipment maycommunicate with the same or with different satellites 106.

The television receivers 116 e, as described throughout, may generallybe any type of television receiver (such as an STB (set-top box), forexample) configured to decode signals received for output andpresentation via a display device 160. In another example, televisionreceiver 116 e (which may include another remote television receiver 116e) may be integrated as part of or into a television, a DVR, a computingdevice, such as a tablet computing device, or any other computing systemor device, as well as variations thereof. In some embodiments, atelevision receiver 116 e may be a component that is added into thedisplay device 160, such as in the form of an expansion card. Atelevision receiver 116 e and network 120 together with televisionreceivers 116 e and/or one or more computing devices 116, may form atleast a portion of a particular home computing network, and may each berespectively configured such as to enable communications in accordancewith any particular communication protocol(s) and/or standard(s)including, for example, TCP/IP (Transmission Control Protocol/InternetProtocol), DLNA/DTCP-IP (Digital Living Network Alliance/DigitalTransmission Copy Protection over Internet Protocol), HDMI/HDCP(High-Definition Multimedia Interface/High-Bandwidth Digital ContentProtection), etc. While only a limited number of television receivers116 e, display devices 160, computing devices 116, etc. are illustrated,it should be understood that multiple (e.g., tens, thousands, millions)instances of such equipment, corresponding to various users in variousgeolocations, may be included the system 200.

In some embodiments, broadcast television programs may be delivered totelevision receivers, including a television receiver 116 e, viasatellite according to a schedule. On-demand content may also bedelivered to a television receiver 116 e via satellite. Satellites 106may be configured to receive uplink signals 122 from satellite uplink104. In this example, uplink signals 122 may contain one or moretransponder streams of particular data or content, such as particulartelevision channels, each of which may be supplied by content providersystem 102-1. For example, each of uplink signals 122 may containvarious media content such as HD (High Definition) television channels,SD (Standard Definition) television channels, on-demand programming,programming information (e.g., table data), and/or any other content inthe form of at least one transponder stream, and in accordance with anallotted carrier frequency and bandwidth. In this example, differentmedia content may be carried using different satellites of satellites106. Further, different media content may be carried using differenttransponders of a particular satellite (e.g., satellite 106-1); thus,such media content may be transmitted at different frequencies and/ordifferent frequency ranges. For example, a first television channel anda second television channel may be carried on a first carrier frequencyover a first transponder (as part of a single transponder stream) ofsatellite 106-1, and a third, fourth, and fifth television channel maybe carried on a second carrier frequency (as part of another transponderstream) over a transponder of satellite 106-3, or, the third, fourth,and fifth television channel may be carried on a second carrierfrequency over a second transponder of satellite 106-1, etc.

The satellites 106 may be further configured to relay uplink signals 122to the satellite receiver 108 as downlink signals 124. Similar to theuplink signals 122, each of the downlink signals 124 may contain one ormore transponder streams of particular data or content, such as variousencoded and/or at least partially scrambled television channels,on-demand programming, etc., in accordance with an allotted carrierfrequency and bandwidth. The satellite receiver 108, which may include asatellite dish, a low noise block (LNB), and/or other components, may beprovided for use to receive television channels, such as on asubscription basis, distributed by the content provider system 102-1 viathe satellites 106. For example, the satellite receiver 108 may beconfigured to receive particular transponder streams as downlink signals124, from one or more of the satellites 106. Based at least in part onthe characteristics of a given television receiver 116 e and/orsatellite receiver 108, it may only be possible to capture transponderstreams from a limited number of transponders of the satellites 106concurrently. For example, a tuner of the television receiver 116 e mayonly be able to tune to a single transponder stream from a transponderof a single satellite, such as the satellite 106-1, at a time.

The television receiver 116 e, which may be communicatively coupled tothe satellite receiver 108, may subsequently select, via a tuner,decode, and relay television programming to a television for displaythereon. Broadcast television programming or content may be presented“live,” or from a recording as previously stored on, by, or at thetelevision receiver 116 e. For example, an HD channel may be output to atelevision by the television receiver 116 e in accordance with theHDMI/HDCP content protection technologies. Other embodiments arepossible. For example, in some embodiments, an HD channel may be outputto a television in accordance with the MoCA® (Multimedia over CoaxAlliance) home entertainment networking standard. Other embodiments arepossible.

The television receiver 116 e may select via tuner, decode, and relayparticular transponder streams to one or more of television receivers116 e, which may in turn relay particular transponder streams to one ormore display devices 160. For example, the satellite receiver 108 andthe television receiver 116 e may, respectively, be configured toreceive, decode, and relay at least one television channel to atelevision by way of a television receiver 116 e. Similar to theabove-example, a television channel may generally be presented “live,”or from a recording as previously stored by the television receiver 116e, and may be output to the display device 160 by way of the televisionreceiver 116 e in accordance with a particular content protectiontechnology and/or networking standard. Other embodiments are possible.

In various embodiments, the content resources may be used to provide thetelevision receiver 116 e with content (e.g., television programming).The content resources may be used to retrieve television programs orportions of television programs following and/or during a live broadcastof the television programs. The content resources may include thecontent provider system 102-1, the service provider systems 103-1, theonline content sources 112, one or more other television receivers 116e, and/or the like.

The content provider system 102-1, which may distribute broadcasttelevision programming to the television receivers 116 e via asatellite-based television programming distribution arrangement (or someother form of television programming distribution arrangement, such as acable-based network, fiber-based network, or IP-based network), may usean alternate communication path, such as via the network 120, to providetelevision programming to the television receivers 116 e. The televisionreceivers 116 e may be permitted to request various television programsor portions of television programs from the content provider system102-1 via the network 120. For instance, the content provider system102-1 may be permitted to transmit a portion of a television program oran entire television program during and/or after a time at which thetelevision program was broadcast live by the television service providervia a satellite-based television programming distribution arrangement.

In some embodiments, the content provider system 102-1 may provide atelevision program via on-demand content. On-demand content may beincluded in a user's subscription or may be provided on a per-requestbasis. Such on-demand content may be provided via the satellite-baseddistribution arrangement and/or via the network 120. On-demand contentprovided via the satellite-based distribution arrangement may be storedlocally by the television receiver 116 e to allow on-demand access.On-demand content may also be retrieved via the network 120 from thecontent provider system 102-1.

The computing devices 116 a-d represent various computerized devicesthat may or may not be associated with a user of the television receiver116 e and that may be configured to facilitate various adaptive contentfeatures disclosed in various embodiments herein. As indicated by 116 a,the computing devices 116 a-d may include a laptop computer, a desktopcomputer, a home server, or another similar form of computerized device.As indicated by 116 b and 116 c, the computing devices 116 a-d mayinclude a cellular phone and/or smartphone, a tablet computer, oranother similar form of mobile device. As indicated by 116 d, thecomputing devices 116 a-d may include smart glasses or another similarform of wearable computing device.

In various embodiments, the television receiver 116 e may be providedwith access credentials that allow access to content stored and/oraccessible through one or more of the computing devices 116 a-d.Likewise, in various embodiments, one or more of the computing devices116 a-d may be provided with access credentials that allow access tocontent stored and/or accessible through the television receiver 116 eand/or account associated therewith and/or associated with anapplication installed on the one or more of the computing devices 116a-d. It should be understood that computing devices 116 a-d areexemplary in nature. Content may be accessible through a lesser orgreater number of computerized devices associated with a user of thetelevision receiver 116 e.

In some embodiments, the online content sources 112 may representcontent resources through which content may be retrieved by the endpointmedia devices 116 via the network 120. Content available through theonline content sources 112 may be available for free and not requiresubscription (a username and/or password may or may not be necessary).Each of the online content sources 112 may represent different websitesavailable via the Internet. For example, some content may be legallymade available for free (such as television programming provided bygovernment-funded sources, e.g., the BBC or Hulu®). Periodically, theendpoint media devices 116 may poll online content sources 112 todetermine which content is available and/or which content is scheduledto be available in the future. In some embodiments, the endpoint mediadevices 116 may poll online content sources 112 regarding theavailability of at least a portion of a specific show.

The service provider systems 103-1 may correspond to one or more datasources 112 that are any suitable source of data to facilitateembodiments disclosed further herein. In various embodiments, theservice provider systems 103-1 may include one or more computer systems,a database, a website, a portal, any repository of data in any suitableform, a server system, and/or the like. With some embodiments, the datasources 112 may include one or more mobile computing device locatorservices that provide information regarding the location of one or morecomputing devices 116 a-d. With some embodiments, the data sources 112may provide various details relating to IP addresses, cellular toweridentification and location data, mobile device triangulation data, LANidentification data, Wi-Fi identification data, access pointidentification and location data, and/or the like data that facilitateslocation of one or more computing devices 116 a-d. With someembodiments, the data sources 112 may provide demographic data about anarea encompassing the location of one or more computing devices 116 a-d.

In various embodiments, the data from the one or more data sources 112may be retrieved and/or received by the content provider system 102-1and/or the subsystem(s) 111 via one or more data acquisition interfacesthrough network(s) 120 and/or through any other suitable means oftransferring data. In various embodiments, the content provider system102-1 and/or the subsystem(s) 111 and the data sources 112 could use anysuitable means for direct communication. The one or more dataacquisition interfaces may include one or more application programminginterfaces (APIs) that define protocols and routines for interfacingwith the data sources 112. The APIs may specify application programminginterface (API) calls to/from service provider systems 103-1. In someembodiments, the APIs may include a plug-in to integrate with anapplication of a service provider system 103-1. The data acquisitioninterfaces, in some embodiments, could use a number of API translationprofiles configured to allow interface with the one or more additionalapplications of the data sources to access data (e.g., a database orother data store) of the data sources 112. The API translation profilesmay translate the protocols and routines of the service provider system103-1 to integrate at least temporarily with the system and allowcommunication with the system by way of API calls. Data, as referencedherein, may correspond to any one or combination of raw data,unstructured data, structured data, information, and/or content whichmay include media content, text, documents, files, instructions, code,executable files, images, video, audio, and/or any other suitablecontent suitable for embodiments of the present disclosure.

Although FIG. 2 illustrates an example that includes a satellite-basedtelevision channel distribution system, it should be understood that atleast some of the aspects of such a system may be similar to a cabletelevision distribution system, an IPTV content distribution system,and/or any other type of media or content distribution system. Forexample, in a cable television system, rather than using satellitetransponders, multiple RF channels on a cable may be used to transmitstreams of television channels. As such, aspects detailed herein may beapplicable to cable television distribution systems.

Further, the content provider system 102-1 may include one or morecontent server systems configured to stream television programming,including televised events such as sports events, to the computingdevices 116 via the network 120. When the streaming content serversstream content to the computing devices 116, the stream content may beprocessed and displayed by the computing devices 116 using one or moreapplications installed on the computing devices 116. Some such streamingservices may require a subscription and may require user authentication,e.g., with a username and/or password which may or may not be associatedwith an account map to the television receiver 116 e. Accordingly, thestreaming services may make a television program available for streamingor download during the live broadcast of the television program.

The television receiver 116 e may be able to retrieve at least a portionof a television program through other television receivers 116 e, whichcan function as content resources. Similarly, the television receiver116 e may be able to cast at least a portion of a television programthrough other television receivers 116 e and/or to computing devices116. For instance, a Slingbox® (or other form of media streaming device)functioning in concert with a television receiver 116 e may permittelevision programs to be captured and streamed over the network 120. Insome embodiments, the television receivers 116 e may have such mediastreaming capabilities integrated. In some embodiments, the televisionreceivers 116 e may cast programming content to the computing devices116 via wireless signals. For example, the programming content from thetelevision receiver 116 e may be indirectly transmitted via a localnetwork (e.g., via Wi-Fi) or directly transmitted to the computingdevice 116 via a casting device integrated with the television receiver116 e or coupled to the television receiver 116 e (e.g., via a dongle).In some embodiments, the programming content may be cast to thecomputing device 116 via a wired connection (e.g., via one or more ofHDMI, USB, lightning connector, etc.). Some embodiments of thetelevision receivers 116 e may provide for simulcasting such that thesame programming that is being displayed on the display device 160 isbeing displayed on one or more of the computing devices 116simultaneously or substantially simultaneously.

A user may be able to obtain at least a portion of a television programvia the television receivers 116 e, which may be associated with otherusers or with the same user. For instance, the user may have multipletelevision receivers 116 e at different locations. Periodically, thetelevision receiver 116 e may poll the other television receivers 116 eto determine which television programs are available and/or whichtelevision programs are scheduled to be available in the future. In someembodiments, the television receiver 116 e may poll the televisionreceivers 116 e regarding the availability of at least a portion of aspecific television program.

Thus, while network configuration data may be broadcast repeatedly viasatellite to television receivers 116 e, it should be understood that asimilar arrangement may be used in cable-based television programmingbroadcast networks to broadcast network configuration. For any of thevarious type of network, various other forms of data may be transmittedvia an Internet-based network 120 connection rather than using thetelevision service provider's proprietary network. For instance, EPGdata may be transmitted to television receivers via the network 120(e.g., Internet) connection. As another example, firmware and/orsoftware updates may be transmitted on demand to a television receivervia the Internet rather than the television receiver receiving theupdate via the television programming broadcast network.

The content provider system 102-1 may include one or more adaptablecontent item 176 repositories and/or particular content item 177repositories. The content provider system 102-1 may store adaptablecontent items 176 and/or particular content item 177 in the one or morerepositories. The one or more repositories may be implemented in variousways. For example, one or more data processing systems may storeadaptable content items. One or more relational or object-orienteddatabases, or flat files on one or more computers or networked storagedevices, may store adaptable content items. In some embodiments, acentralized system stores adaptable content items; alternatively, adistributed/cloud system, network-based system, such as beingimplemented with a peer-to-peer network, or Internet, may storeadaptable content items.

Adaptable content items 176 and/or particularized content items 177 maycorrespond to any one or combination of raw data, unstructured data,structured data, information, and/or content which may include mediacontent, text, documents, files, instructions, code, executable files,images, video, audio, audio video, and/or any other suitable contentsuitable for embodiments of the present disclosure. For example, thecontent items 176 may be adaptable content items that may correspond tovisual and/or audiovisual announcements with graphical and/or audiocomponents particularized to certain types of products and/or services.In some embodiments, the announcements may correspond to commercials tobe presented during commercial breaks of television programming, such astelevised events. In some instances, the content items 176 may besourced by one or more of the service provider systems 103-1.

In some embodiments, the adaptable content items 176 may correspond tovideo and/or audio video file structures with one or more transitionpoints, hooks, frames, windows, and/or the like for merging with one ormore particularized content items, content items 177, particularized tocertain products and/or services. As disclosed herein, the adaptablecontent items 176 may be merged, blended, joined, overlaid, customized,and/or the like in any suitable manner with other particularized contentitems 177 in order to create electronic content composites 180particularized to certain types of products and/or services. In variousembodiments, as further disclosed herein, the adaptable content items176 and/or the other content items 177 may be formatted, rescaled,cropped, image characteristic (e.g., color, brightness, transparency,opaqueness, contrast, etc.) adjusted, and/or otherwise prepared tofacilitate the merging, blending, joining, overlaying, customizing,and/or the like and presentation by endpoint media devices 116 asdisclosed herein.

In various embodiments, the particularized content items 177 maycorrespond to content that is particularized to certain types ofproducts and/or services and that, in various embodiments, may besourced by one or combination of the system 102, one or more of theservice provider systems 103-1, and/or the endpoint media device 116(e.g., in instances where personalization data that may form at leastpart of a particularized content item 177). In various embodiments, theservice provider systems 103-1 may correspond to one or more sources ofdata and/or services corresponding to the adaptable content items 176and/or the particularized content items 177, and particularized contentitems 177 may correspond to the specific data and/or services sourced bya specific service provider system 103-1. For example, the data maycorrespond to particular digital content that is redeemable to discountproduct(s) and/or service(s) (i.e., a redeemable content item, such asan ecoupon), matrix code such as a QR code, trailer, movie content,and/or the like. In some embodiments, particularized content items 177may include personalized content that is particular to one or moreviewers. In some embodiments, the personalized content may be separatefrom the particularized content items 177. As disclosed above, theadaptable content items 176 and/or the content items 177 may correspondto any one or combination of raw data, unstructured data, structureddata, information, and/or content which may include media content, text,documents, files, instructions, code, executable files, images, video,audio, audio video, and/or any other suitable content suitable forembodiments of the present disclosure.

In various embodiments, content items 176 and/or 177 may be activelygathered and/or pulled from one or more data sources 112, for example,by accessing a repository and/or by “crawling” various repositories.Additionally or alternatively, the content provider system 102-1 and/orthe subsystem(s) 111 may wait for updates from one or a combination ofthe content source systems 112. Content items 176 and/or 177 pulledand/or pushed from the one or more data sources 112 may be transformed,and the transformed content items 176 and/or 177 and/or other datagenerated based thereon may be made available by the content providersystem 102-1 and/or the subsystem(s) 111 for use by the subsystem(s) 111in conjunction with content composites 180.

In some embodiments, the content provider system 102-1 may provide theadaptable content items 176 and, in some embodiments, the particularizedcontent items 177, to the endpoint media device 116 as part of a datatransfer that is sent through the satellite 106. For example, in someembodiments, the television receiver 116 e may receive a downlinkedsatellite signal that includes the data for adaptable content items 176and/or particularized content items 177 transferred on a designated linein the vertical blanking interval (VBI) or other portion of thebroadcast service transmission that does not interfere with othertransmitted content. Additionally or alternatively, the content providersystem 102-1 may provide adaptable content items 176 and/orparticularized content items 177 to the television receiver 116 e viathe one or more data networks 120. In some embodiments, the contentprovider system 102-1 may store the adaptable content items 176 and/orparticularized content items 177 in an adaptable content item 176repository and/or a particularized content items 177 repository includedin or otherwise accessible to the content provider system 102-1. In someembodiments, the television receiver 116 e may store the adaptablecontent items 176 and/or particularized content items 177 in anadaptable content item 176 repository and/or a particularized contentitems 177 repository included in the television receiver 116 e orotherwise local to the television receiver 116 e. Consequently, thetelevision receiver 116 e may use one or more of the adaptable contentitems 176 and one or more of the particularized content items 177 inaccordance with embodiments disclosed herein. In some embodiments,additionally or alternatively to providing the adaptable content items176 and/or particularized content items 177, the content provider system102-1 may provide content composites 180 to the television receiver 116e through such means.

The content provider system 102-1 may include a content compositesubsystem 111 in whole or in part. In some embodiments, additionally oralternatively, one or more of the endpoint media devices 116 may includea content composite subsystem 111 whole or in part. The extent to whichthe media devices 116 may be configured to provide features of thesubsystem 111 (e.g., by way of software updates and communications fromthe system 102-1) may depend on the processing power and storagecapabilities of a given device 116. The system 102-1 may communicatewith a given device 116 to pull specifications and current devicecapability information from the device 116. Based on suchcommunications, the system 102-1 may the extent to which the device 116can be configured to provide features of the subsystem 111 and mayoperate accordingly. For example, the system 102-1 may push one or moresoftware packages to the device 116 to configure the device 116 toprovide a set of one or more features of the subsystem 111. In instanceswhere the device 116 lacks sufficient processing and/or storagecapabilities, the subsystem 111 may operate on the system 102-1. As oneexample with respect to many features disclosed herein, the filtering ofcomposites 180 may be performed on the backend at system 102-1 when thedevice 116 lacks sufficient resources to perform the filtering itself.Further, in some embodiments, additionally or alternatively, one or moreservice provider systems 103-1 may include a content composite subsystem111 in whole or in part. The content composite subsystem 111 may beconfigured to facilitate various content adaptation features inaccordance with various embodiments disclosed herein.

The system 102 with the content composite subsystem 111 may beconfigured to perform one or more methods for facilitating adaptivecontent items for delivery in a packet stream, such as methods forcontainerizing and adapting content items, as disclosed herein. Invarious embodiments, part or all of the method may be performed while anendpoint media device 116 is receiving programming content and/or isoutputting programming content for display. In various embodiments, atleast part of the method may be performed in advance of the programmingcontent and, thus, may be performed while is scheduled to outputprogramming content but before the programming content is broadcast toendpoint media devices 116 and/or before the programming content outputby an endpoint media device 116 for display. In various embodiments, oneor more media devices (e.g., the devices 116 a-e and/or the system102-1) may perform all or part of the method, with a single media deviceor multiple media devices performing the method.

In various instances, a media device may receive and process anelectronic communication from a user interface, the electroniccommunication corresponding to an indicator of content. For example, theindicator may correspond to a selection corresponding to a televisedevent, may correspond to an initialization/powering up of the device116, a channel and/or stream selection such as a selection to tune to achannel that is streaming the televised event or that is scheduled thestream the event, an application selection such as a selection todownload or otherwise stream the televised event which may be by way ofan application device, a selection to view and/or record a particulartelevised event, and/or the like.

The media device may receive first content corresponding to theprogramming content and may output the content for display with adisplay device 160 and/or with a display component of a device 116. Thefirst content may be determined to correspond to a televised event asincluding the televised event, as preceding the televised event, and/oras being delivered within a temporal proximity to the televised event.This may include identifying one or more specifications of theprogramming content from the programming content; metadata associatedwith the programming content; EPG or other schedule data received by thedevice 116 from the content provider system 102-1 and mapping such datato the programming content, channel, and/or current or upcomingtime/time period; and/or the like. Some embodiments may further includeidentifying the programming content as likely be viewed by a particularviewer based at least in part on viewer pattern data, even though theviewer has not yet made a selection to view and/or record theprogramming content. For example, as disclosed herein, the pattern datamay indicate a preference for a particular type of programming content.The subsystem 111 may determine that the programming content correspondsto the preference and that temporal specifications for the programmingcontent satisfy one or more temporal thresholds. In some instances, thesubsystem 111 may determine that the programming content is currentlyongoing and available for viewing on another channel, stream, or otherviewing options that the viewer has not yet selected. Likewise, in someinstances, the subsystem 111 may determine that the programming contentis scheduled to be available within a suitable time for threshold (e.g.,a number of minutes, hours, days, weeks, and/or the like) for viewing onthe same channel, stream, or other viewing option that the viewer hasselected or on a different channel, stream, or other viewing option thatthe viewer has not yet selected.

One or more content composites 180 may be created to facilitate deliveryof the composite 180 and display of one or more content items 176 (insome instances, as modified with particular items 177) in conjunctionwith the televised event or other type of audiovisual content (movies,shows, etc.). One or more indicators of one or more state changes withrespect to the content (e.g., event that is televised, show, etc.) maybe detected. The one or more state changes may include upcoming statechanges, such as commercial breaks upcoming within a time threshold(e.g., a number of seconds and/or minutes). In some embodiments, the oneor more content composites 180 may be created and/or deliveredconsequent to the detecting the one or more state changes. In someembodiments, the one or more content items 176 may be adapted with oneor more particular items 177 consequent to the detecting the one or morestate changes.

In some embodiments, the creating the content composite 180 and/or theadaptation of the one or more content items 176 with one or moreparticular items 177 may be a function of a current geolocation of theendpoint media device 116, with a set of rules mapped to the currentgeolocation and specifying geo-specific criteria for creating contentcomposites 180, selecting content items 176 and particularized contentitems 177, adapting the content items 176 with particularized contentitems 177, and provisioning the content items 176 and particularizedcontent items 177. The current geolocation of the media device may bedetermined at a time when the first content is being output for display.With the content composite 180 created and, in some instances, the oneor more content items 176 adapted, the one or more content items 176corresponding to the programming content may be output for display,e.g., during the commercial break, where the content items 176 andparticularized content items 177 are selected based at least in part onlocation metadata mapped to the content items 176 and particularizedcontent items 177 specifying location indicia for the content items 176and particularized content items 177. Some sets of rules may specify athreshold distance, and the content items 176 and particularized contentitems 177 may be selected when a distance between the currentgeolocation of the endpoint media device 116 and the location indiciafor the content items 176 and particularized content items 177 satisfiesthe distance threshold. Some sets of rules may specify that only certaintypes of particularized content items 177 may be selected when thedistance threshold is not satisfied (e.g., promo codes but notecoupons).

Disclosed embodiments may confirm and verify that a content item 176 (insome instances, as modified with one or more particular content items177, such as a redeemable content item, matrix code such as a QR code,particularization data, etc.) was transmitted to an endpoint mediadevice 116. Disclosed embodiments may confirm and verify that themodified or unmodified content item 176 was presented with a displaydevice at the endpoint media device 116 (which display device may beintegrated with the endpoint media device 116 or otherwisecommunicatively coupled to the endpoint media device 116). Disclosedembodiments may confirm and verify to what extent the modified orunmodified content item 176 was presented (in full or to a lesserextent). Disclosed embodiments may confirm and verify that the modifiedor unmodified content item 176 was shown to an intended end user if andwhen utilizing encryption and/or an encryption flag. Disclosedembodiments may confirm and verify detection of one or more operationsexecuted consequent to the presentation (e.g., scanning a matrix code,loading a redeemable content item and/or another particularized contentitem 177 into mobile app such as a digital wallet of the endpoint mediadevice 116, utilizing the particularized content items 177 such as toorder a product and/or service, etc.) and metrics of the operations(e.g., time of execution). Such confirmations may take the form ofreturn channel notifications from the endpoint device 116 to the system102, which may contribute to the observation data 229 and feedback loopfeatures disclosed further herein. In addition to such communicationsfrom endpoint devices 116, the system 102 may receive pushed and/orpulled notifications from one or more service provider systems 103 thatindicate when particularized content items 177 have been used by theendpoint devices 116 with respect to the systems 103 (e.g., to a system103, uploading at least a portion of a content item 177; communicatingat least a portion of a content item 177 such as code, accessing using aURL of a content item 177; transferring, presenting, or otherwiseproviding an image capture of a content item 177; and/or the like). Suchnotifications may also contribute to the observation data 229 andfeedback loop features.

FIG. 3 illustrates a functional diagram of an adaptive content compositesystem 300, in accordance with disclosed embodiments of the presentdisclosure. In various embodiments, the content composite system 300 maybe included in whole or in part in the content provider system 102-1and/or an endpoint media device 116. In some embodiments, the contentcomposite system 300 may be separate from, and provide content to, thecontent provider system 102-1. In some embodiments, the contentcomposite system 300 may be included in the end-user system and may beincluded in the television receiver 116 e and/or one or more of thecomputing devices 116. In some embodiments, various features of thecontent composite system 300 may be distributed between the televisionreceiver 116 e and upstream of the television receiver 116 e. Likewise,in some embodiments, various features of the content composite system300 may be distributed between one or more of the computing devices 116and upstream of the one or more computing devices 116. While not allcomponents of the adaptive content composite system 300 are shown, thesystem 200 may include one or a combination of such components.

As depicted, the content composite system 300 includes a contentcomposite subsystem 111. The content composite subsystem 111 may includeor otherwise correspond to an audiovisual control engine that, as withdisclosed embodiments of the other engines, may include instructionsretained in processor-readable media and to be executed by one or moreprocessors. The content composite subsystem 111 may be communicativelycoupled with interface components and communication channels (e.g., ofthe television receiver 116 e and/or the computing device 116, which maytake various forms in various embodiments as disclosed herein)configured to receive programming content 202, which may correspond totelevised sporting events, movies, television programs, portionsthereof, etc. In various embodiments, the programming content 202 mayinclude audiovisual content broadcast and/or otherwise transmitted bythe content provider system 102-1 and/or one or more other serviceproviders 103-1. The programming content 202 may include variouscomponents, including without limitation, one or more video tracks,audio tracks, audio video tracks, metadata tracks, close captioninginformation, and/or the like. In some embodiments, the content compositesystem 300 may retain received programming content 202 in contentstorage 222. The content storage 222 may include any suitable form ofstorage media, such as any suitable form disclosed herein.

The content composite subsystem 111 may be further configured to receiveadaptable content items 176 and particularized content items 177. Thecontent composite subsystem 111 may include a harvesting engine 236configured to aggregate adaptable content items 176, particularizedcontent items 177, and/or programming content 202 in order to facilitatecontent splicing features disclosed herein. The content compositesubsystem 111 may include a matching engine 238, which, in variousembodiments, may be configured to analyze, classify, categorize,characterize, tag, and/or annotate adaptable content items 176,particularized content items 177, and/or programming content 202.

The content composite subsystem 111 may include a content splicingengine 242. In some embodiments, the content splicing engine 242 mayinclude a multiplexer. In various embodiments, the multiplexer maycreate a digital stream of data packets containing the video, audio,and, in some embodiments, the metadata to output the programming content202, adaptable content items 176, and/or the composites 180 created withselected adaptable content items 176. In various embodiments, thecontent splicing engine 242 may be implemented at the receiver 116 e,the device 116, and/or the service provider system 102-1.

In some embodiments where the content splicing engine 242 is implementedat the service provider system 102-1, the multiplexed data stream may betransmitted via the one or more networks 124 for provisioning tocomputing devices 116 or via a particular transponder stream via atransponder of a satellite four provisioning to receivers 116 e. Themultiplexer may create a digital stream of data packets containing thevideo, audio, and entitlement control messages (ECMs), to be transmittedon the transponder data stream. The data stream, which includes videoand/or audio data packets that are not scrambled, may be passed to ascrambling engine, which may use a control word to scramble video oraudio present in a data packet. Some audio and video packets may alsopass through with no scrambling, if desired by the content providersystem 102-1. A control word generator may generate the control wordthat is used by a scrambling engine to scramble the video or audiopresent in the data packet. Control words generated by the control wordgenerator may be passed to a security system, which may be operated bythe television service provider or by a third-party security provider.The control words generated by control word generator may be used bysecurity system to generate an ECM. Each ECM may indicate two controlwords. The control words indicated may be the current control word beingused to scramble video and audio, and the control word that will next beused to scramble video and audio. The security system may output an ECMto the multiplexer for transmission to subscribers' set-top boxes. Eachdata packet, whether it contains audio, video, an ECM, or some otherform of data, may be associated with a particular PID. This PID may beused by the set-top box in combination with a networking informationtable to determine which television channel the data contained withinthe data packet corresponds. Accordingly, the transponder data streamsmay contain scrambled video packet stream and audio packet stream andalso an encrypted ECM packet stream which contains the control wordsnecessary to descramble the scrambled video and audio packets.

In some embodiments, the harvesting engine 236 may be configured toreceive, pull, process, buffer, organize, rank, and/or store adaptablecontent items 176, particularized content items 177, and/or programmingcontent 202. In various embodiments, the content provider system 102-1,the television receiver 116 e, and/or the computing device 116 mayinclude one or more applications to facilitate the subsystem 111analyzing and consolidating data feeds and/or event updates receivedfrom various data sources 112. As an example, data feeds and/or eventupdates may include, but are not limited to, updates (real-time and/orotherwise) and/or continuous data streams received from one or more datasources 112, which may include real-time events related to promotions,redeemable content items, special offerings, discounts, sports eventinformation, Twitter® feeds, Instagram® posts, Facebook® updates,Amazon® updates, and/or the like.

As disclosed above, the adaptable content items 176 may beparticularized to certain products and/or services. In some embodiments,the adaptable content items 176 may correspond to commercials to bepresented during commercial breaks of the programming content 202. Invarious embodiments, the content items 176 may include audiovisualcontent broadcast and/or otherwise transmitted by the content providersystem 102-1. In some embodiments, adaptable content items 176 may bepushed by the content provider system 102-1 to the subsystem 111. Inaddition or in alternative, adaptable content items 176 may be pulled bythe subsystem 111 (e.g., by the harvesting engine 236) from the contentprovider system 102-1.

The particularized content items 177 may correspond to content that isparticularized to certain types of products and/or services and that issourced by one or more of the service provider systems 103-1. In variousembodiments, the service provider systems 103-1 may correspond to one ormore sources of data, products, and/or services corresponding to theadaptable content items 176, and particularized content items 177 maycorrespond to the specific data and/or services sourced by a specificservice provider system 103-1 for one or more specific products and/orservices. As disclosed above, the adaptable content items 176 and/or thecontent items 177 may correspond to any one or combination of raw data,unstructured data, structured data, information, and/or content whichmay include media content, text, documents, files, instructions, code,executable files, images, video, audio, audio video, and/or any othersuitable content suitable for embodiments of the present disclosure.

In various embodiments, sets of one or more adaptable content items 176and/or sets of one or more content items 177 may be transmitted to thesubsystem 111 in batches. For example, sets of one or more adaptablecontent items 176 and/or sets of one or more content items 177 may betransmitted to the subsystem 111 on a periodic or otherwise scheduledbasis. The subsystem 111 may store the adaptable content items 176locally and, subsequently select one or more of the adaptable contentitems 176 when needed for presentation during an upcoming break in theprogramming content 202 and/or when needed for presentation during theprogramming content 202 based at least in part on the subsystem 111determining specifications of the programming content 202, a temporalprogression in the programming content 202 (e.g., the fourth quarter,the second round, etc.), a state change in the event (e.g., a scorechange, one team or competitor leading or falling behind, etc.), and/orthe like.

In various embodiments, sets of one or more adaptable content items 176and/or sets of one or more content items 177 may be transmitted to thesubsystem 111 on an as-needed basis when the subsystem 111 is receivingprogramming content 202 corresponding to a certain type (e.g., atelevised sporting event), is scheduled to receive such programmingcontent 202, is predicted to receive programming content 202 based atleast in part on a detected viewing pattern of past viewing of previousprogramming content 202 (e.g., of a certain type of event, at certaintimes, on certain days, etc.), and/or is predicted to receiveprogramming content 202 based at least in part on a detected pattern ofpast viewer responses to content composites for previous programmingcontent 202 of that type. Additionally or alternatively, in someembodiments, sets of one or more adaptable content items 176 and/or setsof one or more content items 177 may be selected (e.g., the serviceprovider system 102-1) as tailored for particular event viewing habits,ordering patterns, and inferred interests of viewers.

In various embodiments, sets of one or more adaptable content items 176may be selected (e.g., by the service provider system 102-1) forparticular time periods and may be transmitted to the subsystem 111 withan assignment (e.g., by way of tag data or other metadata) for thedesignated time period. Additionally or alternatively, in someembodiments, sets of one or more adaptable content items 176 may beselected (e.g., by the service provider system 102-1) for particularchannels and/or television programs and may be transmitted to thesubsystem 111 with an assignment (e.g., by way of tag data or othermetadata) for the designated channels, shows, movies, and/or televisionprograms. The transmission of the sets of one or more adaptable contentitems 176 may be in response to the subsystem 111 pulling the sets ofone or more adaptable content items 176 from the service provider system102-1. For example, the subsystem 111 may pull adaptable content items176 based at least in part on detecting programming content 202currently being viewed via a television receiver 116 e or computingdevice 116, detecting programming content 202 scheduled to be viewed orrecorded, predicting programming content 202 of interest to a viewerbased on detected viewing patterns and/or patterns of interacting withcontent items 176, 177, determining upcoming programming content 202based on electronic programming guide information received, and/or thelike.

In a similar manner, sets of one or more content items 177 may be pulledfrom or pushed by one or more service provider systems 103-1, in variousembodiments using one or more of the various methods disclosed, to thesubsystem 111 directly or indirectly (e.g., by way of the contentprovider system 102-1, which may then transmit the content items 177 tothe subsystem 111) for particular time periods, with assignments fordesignated channels, shows, movies, and/or television programs. Forexample, in conjunction with the selection of sets of one or moreadaptable content items 176, sets of one or more content items 177 thatmatch the one or more adaptable content items 176 may be pulled from oneor more service provider systems 103-1. In various examples, thematching may be based at least in part on specifications of the event, atemporal progression in the event (e.g., the fourth quarter, the secondround, etc.), a state change in the event (e.g., a score change, oneteam or competitor leading or falling behind, etc.), and/or the like.

The subsystem 111 may use the rules 218 to adaptively control contentcomposite creation, the delivery of such content, and interactions withsuch content. The categorization 212 of the harvested regulations andupdated regulations may be used to create various sets of rules 218governing the creation and provisioning of particularized contentcomposites 180. The various sets of rules 218 may govern the creationand provisioning of particularized content composites 180 as a functionof a location of a particular receiver 116 e and/or device 116. Hence,different sets of rules 218 may apply to different locations. Further,various sets of rules 218 may provide for various types of restrictionsand/or specifications on creating and/or provisioning content composites180. In addition to geolocation restrictions/specifications, the varioustypes of restrictions and/or specifications may include timerestrictions, such as limits on a time of day when content composites180 may be presented, limits on time in advance particular event (e.g.,days, hours, etc.) and/or portion thereof ahead of which contentcomposites 180 may be presented, and the like. Additionally oralternatively, the various types of restrictions and/or specificationsmay include restrictions on and/or specifications of types of content(e.g., ratings for different audience designations, contentdesignations, etc.) for which content composites 180 may or may not bepresented and the manner in which content composites 180 may bepresented for the different types of programs and/or viewers (e.g.,adult-directed ads may not be presented during children-directed showsand/or to viewers identified as children). Thus, provisioning of contentcomposites 180 may be further differentiated according to show typeand/or viewer type, with time, place, and/or mannerrestrictions/specifications contingent on show type and/or viewer type.Restrictions on and/or specifications of the manner in which contentcomposites 180 may be presented may include differentiating types ofdevices (e.g., smart phone versus laptop computer, laptop computerversus television receiver, etc.) which will display the contentcomposites 180.

To facilitate geo-discrimination to differentiate which sets of rules218 apply to a given content provisioning instance with respect to aprogram and which content items 176, 177 to select and present as afunction of the current device 116 location, disclosed embodiments mayprovide for capturing and analyzing location data 179 for the device 116to determine a current location of the device 116. The current locationof the device 116 may also be used to identify, by a system 102, 103and/or an application of the device 116, when a content item 177 storedby the device 116 (e.g., in a digital wallet) is applicable based atleast in part on location metadata of the content item 177 thatspecifies a product/service provider location that is within a thresholddistance from the current location. The threshold distance may bespecified by the location metadata, user preferences, applicationsettings, and/or the system 102, 103. In instances where the thresholdis satisfied, one or more push notifications may be transmitted from oneof the systems 102, 103 to the endpoint media device 116; theapplication of the endpoint media device 116 may be invoked, awakened,opened, otherwise activated; and/or the application may pop up anotification on a display of the endpoint media device 116 to alert theuser to the content item 177 and the proximate provider location.

Location data 179 may also be captured to facilitate geo-sensitiveadaptive content splicing adaptive content splicing with respect tocontent 202 corresponding to a televised event as a function of alocation detected for the device 116 that receives the content and is tocause display of content composites 180 in conjunction with the content.In various embodiments, the matching engine 238 may include a locationcorrelation engine that may correlate location data 179 to a set of oneor more zip codes (or other postal codes) and a corresponding rule setidentifier for a set of one or more rules 218 mapped to the set of oneor more zip codes (or other postal codes) via identifiers keyed with oneor more tables and/or indexes.

In various embodiments, location data may be determined by televisionreceivers 116 e and/or devices 116, and such data may be sent to thesystem 102-1. The television receivers 116 e and/or devices 116 may, insome embodiments, have location detection capabilities based at least inpart on location data provided by way of device GPS capabilities, Wi-Fi,cellular, other access points, subscriber/account information, and/orthe like techniques for determining a current location of the respectivereceiver 116 e and/or device 116, and corresponding location data 179may be transmitted to the system 102-1. In some embodiments, the system102-1 may gather the location data 179. In some embodiments, where thelocation data 179 does not explicitly indicate a geolocation, the system102-1 may determine geo-locations by cross-referencingsubscriber/account identifiers with stored geolocation data associatedwith subscribers/accounts.

Additionally, in some embodiments, the receiver 116 e and/or device 116may be capable of communicating with a Global Positioning System (GPS)in order to determine to location of the respective receiver 116 eand/or device 116. The antenna may be a GPS receiver or otherwiseinclude a GPS receiver. In various embodiments, communication with thereceiver 116 e and/or device 116 may be conducted with a single antennaconfigured for multiple purposes (e.g., cellular, transactions, GPS,etc.), or with further interfaces (e.g., three, four, or more separateinterfaces). In some embodiments, an application installed on thereceiver 116 e and/or device 116 may cooperate with the system 102-1 tofacilitate tracking of locations of the receiver 116 e and/or device116. For example, the receiver 116 e and/or device 116 may transmitlocation data 179 to any suitable backend system component. The locationdata 179 may be a combination of data based on one or a combination ofGPS, Wi-Fi, cellular, device sensor(s) such as a barometric sensor oraccelerometer, RFID device signals, and/or other techniques fordetermining a current location of the receiver 116 e and/or device 116.

The receiver 116 e and/or device 116 may access the one or more networks120 through one or more wireless links to one or more access points. Theaccess points may be of any suitable type or types. For example, anaccess point may be a cellular base station, an access point forwireless local area network (e.g., a Wi-Fi access point), an accesspoint for wireless personal area network (e.g., a Bluetooth accesspoint), etc. The access point may connect the receiver 116 e and/ordevice 116 to the one or more networks 120, which may include theInternet, an intranet, a local area network, a public switched telephonenetwork (PSTN), private communication networks, etc. In someembodiments, access point(s) may be used in obtaining location data 179for the receiver 116 e and/or device 116.

FIG. 4 is a simplified illustration of a composite build engine 240, inaccordance with disclosed embodiments of the present disclosure. Invarious embodiments, the composite build engine 240 may be included inthe subsystem 111 or may be separate from the subsystem 111. Thecomposite build engine 240 may, in some embodiments, be included in thecontent provider system 102-1. Having processed an adaptable contentitem 176, the subsystem 111 may create one or more content composites180 that may include the adaptable content item 176.

To create the content composites 180, disclosed embodiments may providethe content item 176 with containerization objects 182 that eachcontainerize a set of data as a persistent object in time. Eachcontainerization object 182 may be configured to function in a broadcastenvironment while facilitating the various features disclosed herein.For example, among other things, the containerization object 182 maycreate the ability for the disclosed deeper integration with endpointdevices 116 in a broadcast environment. With the containerization object182, the content composites 180 may allow the content item 176 to betreated as an object wrapped with an identity and with one or morelayers of encryption. As disclosed herein, the one or more layers ofencryption provided with various embodiments may correspond toencryption at one or more software layers, as distinguishable fromhardware encryption of a device 116. The various embodiments of one ormore layers of encryption may allow for encryption of one or acombination of return channel communications, data regarding endpointdevice 116 operations that contribute to observation data 229, contentitems 176, content items 177, composites 180, and/or components thereof,which may particularly include encryption of personalization andtransactional data, which may provide for DRM, and which may ensure enduser privacy.

The composite build engine 240 may configure the containerization object182 to include a container flag 184. The container flag 184 may includeone or more parameters that indicate one or more containerizationspecifications 186. The container flag 184 may be thin and may indicateother components packetized in the containerization object 182. Forexample, in various embodiments, container flag 184 may include indiciaof one or a combination of: an identifier of the content item 176 in thecontainer, boundaries of the set of one or more audio and/or videopackets that form the content item 176, access and reference to one ormore other containerization specifications 186, access and reference toother metadata 188, and/or the like. The content item identifier maycorrespond to a field with parameters that identifies the content item176.

Further, the composite build engine 240 may configure thecontainerization object 182 to include the containerizationspecifications 186 to facilitate performance of a set of one or moreoperations by the one or more endpoint media devices 116 with respect tothe content item 176 consequent to the one or more endpoint devices 116receiving the content composite 180. In various embodiments, thecontainerization specifications 186 may include one or a combination ofinstructions, metadata, personalization content or instructions to fetchpersonalization content, and/or the like to specify and facilitateperformance of the set of one or more operations by an endpoint mediadevice 116. In some embodiments, the containerization specifications 186may include a particularized content item 177 or instructions to fetch aparticularized content item 177.

Accordingly, the content composite 180 may include the content item 176and the containerization object 182. Further, in various embodiments,the content composite 180 may include a particularized content item 177at the time the content composite 180 is transmitted to one or moreendpoint media devices 116, separate from or grafted into the contentitem 176 such that the content item 176 is an adapted content item 176adapted with the particularized content item 177. In some embodiments,the content composite 180 may not include a particularized content item177 at the time the content composite 180 is transmitted to one or moreendpoint media devices 116. In such instances, the particularizedcontent item 177 may be fetched per the containerization specifications186 from the system 102, 200, another data source 103, or from storageof the endpoint media device 116, and may be used by an endpoint mediadevice 116 to adapt the content item 176. The containerizationspecifications 186 may include instructions and build specificationsaccording to which the endpoint media device 116 may merge thepulled/retrieved particularized content item 177 with the content item176. As part of the instructions and build specifications, thecontainerization specifications 186 may specify that the particularizedcontent item 177 (e.g., a particular redeemable content item, matrixcode such as a QR code, trailer, movie content, and/or the like) containone or more date and/or time attributes (e.g., a time stamp) indicate ofone or a combination of when the item was created, when the item is tobe output by the endpoint media device 116 for display, when the item isstored by the endpoint media device 116 (e.g., in a digital wallet),and/or the like. In some embodiments, the one or more date and/or timeattributes may be stored with the metadata 188.

In various embodiments, the content composite 180 may further includepersonalization content personalized to the end user of the endpointmedia device 116 at the time the content composite 180 is transmitted toone or more endpoint media devices 116, separate from or grafted intothe content item 176 such that the content item 176 is an adaptedcontent item 176 adapted with the personalization content. In someembodiments, the content composite 180 may not include personalizationcontent at the time the content composite 180 is transmitted to one ormore endpoint media devices 116. In such instances, the personalizationcontent may be fetched per the containerization specifications 186 fromthe system 102, 200, another data source 103, or from storage of theendpoint media device 116, and may be used by an endpoint media device116 to adapt the content item 176 according to instructions and buildspecifications of the containerization specifications 186.

The subsystem 111 may transmit the content composite 180 to one or moreendpoint media devices 116, and, in some embodiments, may broadcast thecontent composite 180 to a plurality of endpoint media devices 116. Thesubsystem 111 may transmit the content composite 180 in accordance withcomposite transmission specifications. The composite transmissionspecifications may at least partially specify transmission parametersthat govern transmission of the containerization object 182 in a datastream with the set of one or more audio and/or video packets in audioand/or video streams corresponding to the content item 176.

The content item 176 may correspond to a specific unit within thecontent stream. The specific unit in the content stream may correspondto a section of the video stream that includes the set of one or moreaudio and/or video packets for the content item 176. The compositetransmission specifications may specify how the containerization object182 should be transmitted in the data stream with the content item 176,how many instances of the containerization object 182 should be createdand transmitted, how should some instances of the containerizationobject 182 be time/place shifted in advance of the packets correspondingto the content item 176, and/or the like. Thus, the compositetransmission specifications may provide for one or more instances of thecontainerization object 182 particularized for the individual contentitem 176 to be transmitted within the data stream of the content stream,with the content item 176 transmitted in the audio and/or videostream(s). Accordingly, a number of instances of the containerizationobject 182 with the content item identifier, as specified by thecomposite transmission specifications, may be embedded in a section ofthe data stream that indicates the content item 176. Not only may thecontainerization object 182 with the container flag 184 be carried as adata stream with the audiovideo content stream, flagging the contentitem 176, it may also be replicated and encoded in a carousel manner perthe composite transmission specifications to facilitate pick-upmid-stream of the content item 176, and/or in advance of the contentitem 176 in the stream to facilitate forward identification of theupcoming content item 176.

The container flag 184 and its indicia may be encrypted and transmittedto the endpoint media device(s) 116 in the encrypted state with thecontent composite 180. Additionally or alternatively, thecontainerization specifications 186 may be encrypted and transmitted tothe endpoint media device(s) 116 in the encrypted state with the contentcomposite 180. Accordingly, disclosed embodiments may add an encryptedcontainer flag 184 to the content item 176.

In various embodiments, the encrypted container flag 184 may governaccess by the endpoint media device 116 to one or more other componentsof the content composite 180. Consequent to delivery of the contentcomposite 180 to the endpoint media device 116, the endpoint mediadevice 116 may decrypt the encrypted container flag 184. Once theendpoint media device 116 decrypts the container flag 184, the endpointmedia device 116 may have access to, for example, the metadata of thecontent composite 180, such as the instructions of the containerizationspecifications 186, other containerization specifications 186, and/orthe other metadata 188.

The container flag 184 may include indicia that identifies the state andextent of one or more layers of encryption of the content composite 180.One or more of the components of the content composite 180 may beencrypted. The encryption may include a lightweight, point-to-pointencryption for specific components of the content composite 180, whichmay be advantageous to avoid bogging down processing in view ofrelatively heavier processing needs for video of content items 176 andgenerally shorter durations of content items 176.

The indicia may flag one or more objects and/or containers for thespecific unit(s) within the content stream without the entire datastream being encrypted. The subsystem 111 may all or part of contentcomposite 180 contingent on the extent of the personalization of thecontent composite 180. For example, when the content composite 180 istransmitted with personalization content, the personalization contentmay be encrypted. As one example out of many possibilities, when acontent item 176 is adapted with a list of items to acquire prepared foran identified viewer, at least the portion of data corresponding to thelist may be encrypted, whereas the balance of the content item 176 thatis not personalized may not encrypted. Accordingly, a partiallycustomized content composite 180 and/or content item 176 may only bepartially encrypted. However, a highly customized content composite 180and/or content item 176 may be entirely or substantially entirelyencrypted, commensurate with the extent of the customization for theidentified viewer. For example, if the content item 176 is due to anidentified viewer selection (say, a prior procurement, such as theviewer having obtained access permissions to a movie or other particularcontent), then, after scraping an ID that a viewer has allowed such, thecontent composite 180 and/or content item 176 (which, for example, maybe directed to opening up 1-to-1 connection to access a digital ticket,access to a digital copy of the content, bonus content, a content itemthat requires tracking such as a redeemable content item for apay-per-view, etc.) may be entirely or substantially entirely encrypted.Such content may be highly customized and may include watermarking. Oneor more particularized content items 177, for example, may includewatermarks for merging with content items 176 as disclosed herein.

The content composites 180 may provide for advantages with greaterlayers within a content stream more based around computer language. Insome embodiments, various options for content items 176 may bebroadcasted, with only a subset of the content items 176 being presentedby particular endpoint media devices 116. The indicia of the containerflags 184 of the content composites 180 may allow for filtering thecontent composites 180 according to attributes of the endpoint mediadevice 116 and/or the endpoint viewer so the matching composite 180 ispresented to the identified viewer/viewer type, pursuant to thedifferentiated interaction with mobile app, etc. For example, flagindicia may include layer flag indicia that indicates this content itemis for unlocked devices or only locked devices. One layer may allow foridentifying different types of the endpoint media devices 116 thatdiffer in capabilities, which, in some embodiments, may be determiningbased at least in part on pulling information (e.g., devicespecifications) from the particular endpoint media device 116. Suchdevice type identification may further allow for selectively providing atype of endpoint media device 116 with different types of content items176 and different content attributes as a function of the device type.For example, such selective provisioning may include filtering contentitems 176 based at least in part on the differences in frame rate of thecontent items 176 as a function of the endpoint media device type and/orcapabilities. Different content items 176 may be filtered for differentendpoint media device 116 (e.g., content items 176 with frame ratesassociated with fast-changing content may not be mapped to devices withlesser capabilities). As another example, such selective provisioningmay include filtering content items 176 based at least in part on thedifferences in how the content items 176 are compressed as a function ofthe endpoint media device type and/or capabilities.

One layer may allow for determining endpoint media device 116authentication. The endpoint media device 116 authentication may bedependent on discovery processes that the determine what is on thedevice that can be used to sufficiently authenticate the endpoint mediadevice 116. For example, the discovery processes may determine andutilize a SIM card of a mobile device, television receiver with asmartcard, and/or similar identification that can be used toauthenticate to set up account. Having determined that the endpointmedia device 116 and/or the endpoint has potential to have an account,another layer may allow for search and authentication operations to tryto find an account associated with the endpoint media device 116 and/orthe endpoint and then authenticate against services related as part of asubscriber account.

Another layer may allow for filtering one or a combination of thecontent composite 180, the content item 176, the particularized contentitem 177, and/or the personalization content based at least in part onthe authentication and identification of the endpoint viewer. Forexample, such filtering may include selecting and presenting the contentcomposite 180, the content item 176, the particularized content item177, and/or the personalization content that correspond to one or moredemographics of the identified viewer (e.g., age, marital status,children, pets, renter, homeowner, etc.) and/or viewer type (e.g., flagindicia of NC-17, MA, or another rating so an endpoint device withparental specifications would be provided with appropriate contentitems). As another example, such filtering may include selecting andpresenting the content composite 180, the content item 176, theparticularized content item 177, and/or the personalization content thatcorrespond to identified past actions mapped to the endpoint and/orendpoint media device 116. This can result in such differences aspresenting content that is directed to, say, “get video subscriptionservice X” versus “because you have video subscription service X, . . .Y.” Another layer may allow for creating a backchannel to facilitatecommunications, ordering, secure video delivery, and/or the likedisclosed herein.

In some embodiments, the container flag 184 and/or containerizationspecifications 186 may prompt the endpoint media device 116 to executethe instructions to perform at least one operation of the set of one ormore operations facilitated by the containerization object 182. In someembodiments, APIs may be used to instruct the endpoint media device 116as to what to do with the container flag 184 and/or containerizationspecifications 186. In some embodiments, the container flag 184 and/orcontainerization specifications 186 may allow for invoking, waking up,opening, and/or otherwise activating an application of the endpointmedia device 116 responsive to the decryption of the container flag 184and/or containerization specifications 186, in some instances, when theapplication is offline and/or otherwise not online with respect to thesystem 102, 200, and/or another system 103.

For example, the container flag 184 and/or containerizationspecifications 186 may include a read flag that triggers one or morereturn channel communications. The corresponding instructions mayinstruct the endpoint media device 116 to report to the system 102, 200,and/or another system 103 with one or more return channel communicationsone or a combination of: that the content item 176 was received by theendpoint media device 116; that the content item 176 was presented witha display device at the endpoint media device 116; to what extent thecontent item 176 was presented (in full or to what lesser extent inpercentage or another suitable metric); that the content item 176 wasshown to an intended endpoint user, detection of one or more operationsexecuted consequent to the presentation (e.g., scanning a matrix code,loading a redeemable content item and/or particularization content intoa mobile app such as a digital wallet of the endpoint media device 116,utilizing the particular content items 177 such as to order a productand/or service, etc.) and metrics of the operations (e.g., time ofexecution); and/or the like. Further, the return channel communicationsmay include one or a combination of: indicia of a particular endpointuser and/or type of endpoint user using the endpoint media device 116within a time period before and/or when at least part of a content item176 is/was output for display; indicia of endpoint media device 116capabilities (e.g., device specifications); indicia of authenticationbases such as a SIM card of a mobile device, television receiver with asmartcard, and/or similar identification that can be used toauthenticate to set up account; and/or the like.

The return channel communications may contribute to the observation data229 and feedback loop features disclosed further herein. Thecontainerization specifications 186 may instruct the endpoint mediadevice 116 to report back upon occurrence of each state change and/or toreport back per a reporting schedule (e.g., just prior to closure of anapplication, at end of day, at the end of another reporting period,etc.). In various embodiments, the container flag 184 and/orcontainerization specifications 186 may include one or more encryptedmessages, hashes, and/or the like that each indicate one or more of theabove reporting events. Accordingly, the instructions may specify thateach return channel notification include a corresponding encryptedmessage, hash, code, and/or the like, which the system 102, 200, and/oranother system 103 may process to verify the respective reporting eventand metrics thereof.

In instances where the system 102 may receive pushed and/or pullednotifications from one or more service provider systems 103 thatindicate when particularized content items 177 have been used by theendpoint devices 116 with respect to the systems 103 (e.g., to a system103, uploading at least a portion of a content item 177; communicatingat least a portion of a content item 177 such as code, accessing using aURL of a content item 177; transferring, presenting, or otherwiseproviding an image capture of a content item 177; and/or the like), thenotifications may contain indicia of one or a combination of the contentitem 177, the endpoint device 116, the user associated with the endpointdevice 116, a location associated with the use of the content item 177,date and/or time associated with the use of the content item 177, and/orthe like. Having received such indicia from a service provider system103, the subsystem 111 (e.g., the matching engine 238) may correlate theindicia to records stored in a database 222 with identifiers of one or acombination of the content item 177, the endpoint device 116, the userand/or account associated with the endpoint device 116, a locationassociated with the user, account, and/or endpoint device 116, and/ordate and/or time associated with the delivery and/or presentation of thecontent item 177. The identified correlations may contribute to theobservation data 229 and feedback loop features.

In some embodiments, where the content item 176 is not already adaptedwith the particularized content item 177, the set of one or moreoperations may include identifying the particularized content item 177and adapting the content item 176 with the particularized content item177. When the particularized content item 177 is not included in thecontent composite 180 when the content composite 180 is received by theendpoint media device 116, the set of one or more operations may includepulling the particularized content item 177 from the system 102, 200,another data source 103 per the containerization specifications 186. Insome embodiments, the particularized content item(s) 177 may betransmitted separately from the system 102, 200 and/or another datasource 103 prior to the delivery of the content composite 180 such thatthe endpoint media device 116 stores the particularized content item(s)177 locally. In such instances, the set of one or more operations mayinclude retrieving the matching particularized content item 177 fromstorage of the endpoint media device 116 per the containerizationspecifications 186. Further, in some embodiments, when theparticularized content item 177 is not transmitted with the content item176, the containerization specifications 186 may specify first checkinglocal storage of the endpoint media device 116 for the matchingparticularized content item 177 and only pulling the particularizedcontent item 177 from the system 102, 200 and/or another data source 103when the particularized content item 177 is not available from the localstorage.

Similarly, the set of one or more operations may include identifying thepersonalized content that is particular to one or more viewers andadapting the content item 176 with the personalized content. Asdisclosed herein, in some embodiments, the particularized content item177 may include personalized content. In such instances, the adapting ofthe content item 176 with the particularized content item 177 mayinclude adapting of the content item 176 with personalized content.Additionally or alternatively, personalized content may be separate fromthe particularized content item 177. Hence, where the content item 176is not already adapted with the personalized content, the personalizedcontent may be included in the content composite 180 when the contentcomposite 180 is received by the endpoint media device 116. When thepersonalized content is not included in the content composite 180 whenthe content composite 180 is received by the endpoint media device 116,the set of one or more operations may include retrieving/pulling thepersonalized content from one or a combination of local storage of theendpoint media device 116, the system 102, 200, and/or another datasource 103 per the containerization specifications 186. Thecontainerization specifications 186 may specify first checking localstorage of the endpoint media device 116 for the matching personalizedcontent and then pulling the personalized content from the system 102,200 and/or another data source 103 when at least some of thepersonalized content is not available (or when sufficient personalizedcontent of one or more specified types is not available) from the localstorage or when supplemental personalized content is identified asrequested per the containerization specifications 186.

To facilitate the content composite 180, the composite build engine 240may include a metadata handler 208 that may generate metadata (e.g., oneor more tags) corresponding to identifiers, attributes, characteristics,and/or categories of programming content 202, adaptable content items176, and/or particularized content items 177. In some embodiments, thecomposite build engine 240 may assign packet identifiers to identifydata of the content that is to be transmitted as part of a data streamto an endpoint media device 116 and that is to be associated with one ormore tags. Data transmitted as part of a data stream to an endpointmedia device 116 may be packetized and assigned packet identifiers. Aparticular packet identifier may be associated with data packets relatedto tags for particular segments of programming content 202, adaptablecontent items 176, and/or particularized content items 177.

The content matching engine 238 may identify a televised event in theprogramming content 202 and may identify one or more correspondingidentifiers, attributes, characteristics, and/or categories ofprogramming content 202, adaptable content items 176, and/orparticularized content items 177 of one or more adaptable content items176 and/or one or more particularized content items 177. Based at leastin part on such identification, the composite build engine 240 maycreate metadata, which, in some embodiments, may correspond to tag data.

Tag data may include an indication of a period of time (or other measureof time, e.g., a number of frames), a start frame, an end frame, and/orthe like. Tag data may include or otherwise be associated with a tagidentifier and may include event, attribute, characteristic, and/orcategory identifiers. For example, the metadata for the televised eventmay identify the particular event. The metadata may further identify oneor more attributes of the particular event (e.g., any suitableidentifier for the participating entities, the location of an event,and/or the like). In some embodiments, at least a portion of themetadata augmentation may be performed at the content provider system102-1 such that one or more tagged composite components may be isprovided to an endpoint media device 116. Subsequently, the endpointmedia device 116 may identify composite components, for example, byprocessing the metadata.

The metadata for adaptable content items 176 may, for example, identifythe adaptable content items 176 as being adaptable with any suitableidentifier, such as a flag, field value, etc. Additionally oralternatively, the metadata for the adaptable content items 176 mayidentify that the adaptable content items 176 are designated for acertain event or category of events with any suitable identifier. Themetadata for the adaptable content items 176 may further identify one ormore attributes of the particular event (e.g., any suitable identifierfor associated entities, location, a temporal attribute such as a timeof an event, and/or the like). Additionally or alternatively, themetadata for the adaptable content items 176 may identify transitionpoints, hooks, frames, windows, other portions designated for overlays,and/or the like for merging with content items 177 such that contentfrom the content items 177 is merged at the transition points, hooks,frames, windows, other portions designated for overlays, and/or thelike.

In some embodiments, metadata-augmented adaptable content items 176 maybe provided by the service provider system 102-1 to the endpoint mediadevices 116, after which the endpoint media devices 116, each having atleast a portion of the content composite subsystems 111, may process anduse the metadata to facilitate matching adaptable content items 176 withcorresponding programming content 202. Likewise, the endpoint mediadevices 116 may process and use the metadata to facilitate matchingadaptable content items 176 with corresponding content items 177. Thus,the metadata may facilitate the endpoint media devices 116 appropriatelyproviding corresponding content composites 180 for display withappropriate placement with respect to televised events, shows, etc. atcommercial breaks and/or during presentation of the televised events,shows, etc.

In a similar manner, metadata for particularized content items 177 may,for example, identify an identifier of the particular event, show, etc.The metadata for the content items 177 may further identify fields andcontent for one or more attributes, such as URLs and hyperlinks toplatforms and websites and/or sites for further information, ordering,and/or the like. In some embodiments, at least a portion of the metadataaugmentation may be performed at the service provider system 102-1and/or the service provider system 103-1 with respect to theparticularized content items 177. Subsequently, the content compositesubsystem(s) 111 and/or endpoint media devices 116 may process and usethe metadata to facilitate matching adaptable content items 176 withcorresponding content items 177. In some embodiments, the devices 116,having at least a portion of the content composite subsystem 111, mayprocess the particularized content items 177 in the form in which theyare received (e.g., directly from a service provider 103-1) and, basedon such processing, may match content items 176 with correspondingcontent items 177.

Referring again more particularly to FIG. 3 , the matching engine 238may be configured to match adaptable content items 176 to segments ofprogramming content 202 based at least in part on metadata at a serviceprovider system 102-1 side or at a device 116, in accordance withvarious embodiments. For example, metadata may be extracted when orbefore a given segment of programming content 202 is to be output fordisplay and before a transition point. In some embodiments, the matchingengine 238 may read the metadata mapped to the segment and perform asearch of the repositories 222 for one or more adaptable content items176 that have metadata matching the extracted metadata with respect toone or more of event identification, event category identification,and/or temporal identification, with the highest preference given to theadaptable content item 176 that has metadata most closely matching themetadata of the previous segment. Alternatively, the matching engine 238may read the metadata mapped to the segment and pull one or moreadaptable content items 176 from the service provider system 102-1. Inso doing, the subsystem 111 may transmit at least a portion of themetadata of the programming content 202 to the service provider system102-1 in order to facilitate matching the extracted metadata with one ormore adaptable content items 176 with respect to one or more of eventidentification, event category identification, and/or temporalidentification. Some embodiments may include the subsystem 111configured to perform a search of the repositories 222 for one or moreadaptable content items 176 that have metadata matching the extractedmetadata in addition to pulling one or more adaptable content items 176from the service provider system 102-1. For example, the subsystem 111may first perform a search of the repositories 222 for any matchingadaptable content items 176 and then only pull one or more adaptablecontent items 176 from the service provider system 102-1 when noadaptable content items 176 are found in the search of the repositories222 that match the extracted metadata with a sufficient match score thatindicates a level of correlation satisfying a correlation threshold.Accordingly, certain embodiments may facilitate matching of adaptablecontent items 176 to segments of programming content 202 based at leastin part on metadata.

In a similar manner, the matching engine 238 may be configured to matchparticularized content items 177 to one or more of content items 176,segments of programming content 202, and/or provider systems 103 at aservice provider system 102-1 side or at a device 116, in accordancewith various embodiments. For example, in some embodiments, the matchingof one or more particularized content items 177 may be based at least inpart on metadata. The metadata may be extracted and read when or beforea given content item 176 and/or a given segment of programming content202-1 is to be output for display, and, before a transition point, maybe mapped to the content item 176 or the segment. In some embodiments,the matching of one or more particularized content items 177 may bebased at least in part on identified characteristics of a given contentitem 176 and/or a given segment of programming content 202. The matchingengine 238 may take the identified characteristics of a given contentitem 176 and/or a given segment of programming content 202 as cues toretrieve one or more content items 177 and/or one or more correspondingadaptable content items 176 and for the content splicing engine 242 tooutput one or more content composites 180 for presentation. The timingof the presentation may be after the segment of programming content 202(e.g., at a commercial break), during the segment of programming content202 (e.g., as an overlay, in a side window/frame, etc.) to match theidentified characteristics of the programming content 202, or after thepresentation of a first composite 180 (e.g., when one or more contentitems 177 and/or one or more corresponding adaptable content items 176are selected for presentation as a subsequent composite 180 or forseparate provisioning to the device 116 in accordance with variousembodiments disclosed herein such as transferring to an accountaccessible by a user of the endpoint media device 116).

In various embodiments, the characteristic identification may includeone or a combination of text recognition, image recognition, and/oraudio recognition. For example, the matching engine 238 may recognizetext from the metadata of content items 176 and/or programming content202. As some possible examples out of many possibilities, the matchingengine 238 may recognize indicia of uniform resource identifiers (e.g.,URLs), identifiers of provider systems 103, and/or the like from themetadata (which may include closed captioning information) that thematching engine 238 may map to provider systems 103. To facilitate imagerecognition, some embodiments of the content composite subsystem 111(e.g., by way of the matching engine 238) may be configured to detectone or more color characteristics in content items 176 and/orprogramming content 202, which may include any suitable colorcharacteristic that may be measured to recognize alpha and/or numericalportions of the images and/or objects in the images. From such portionsand/or objects, the matching engine 238 may recognize uniform resourceidentifiers (e.g., URLs, web addresses, etc.), phone numbers,identifiers of provider systems 103, logos, products, graphics displayedwith streaming tickers or overlays typically positioned and/or scrollingacross a portion of the display area, and/or the like which the matchingengine 238 may map to provider systems 103. The matching engine 238 mayinclude an image analyzer and handling module to facilitate thatdetection. By way of example, the matching engine 238 may detect one ormore color characteristics in programming content 202 by way of any oneor combination of pixel identification, detecting color componentvalues, detecting color codes, and/or the like.

To facilitate audio recognition, the content composite subsystem 111(e.g., by way of the matching engine 238) may be configured to detectone or more audio characteristics in content items 176 and/orprogramming content 202. The matching engine 238 may include an audioanalyzer and handling module to facilitate that detection. By way ofexample, the matching engine 238 may detect one or more audiocharacteristics in content items 176 and/or programming content 202 byway of any one or combination of analyzing audio, applying voicerecognition, acoustic spectrum analysis, analyzing the audio tracksmetadata track, comparison to acoustic profiles for types of content,and/or the like. Some embodiments of the content composite subsystem 111may store acoustic profiles mapped to types of content items 176 and/orprogramming content 202 and corresponding provider systems 103 in therepository 222. Each acoustic profile may specify one or more selectedvalues of acoustic metrics as distinctive markings and/or acousticcategories 214 (e.g., songs, musical scores, jingles, voicecharacteristics, and/or the like) as characteristics of the types ofcontent items 176 and/or programming content 202 and correspondingprovider systems 103.

As another example, the subsystem 111 may recognize one or more keywordsfrom dialogue that the matching engine 238 may map to provider systems103. The matching engine 238 may identify one or more keywords and/orexpressions in the dialogue as a dialogue impression for the purposes ofcharacterizing the sample and identifying content items 176 and/orprogramming content 202 and corresponding provider systems 103. Thematching engine 238 may compile the keywords and/or expressions of thesample and retain the keywords and/or expressions and correspondingprovider systems 103 in the dialogue impression. In some cases, thematching engine 238 may correlate the dialogue impression to one or moredialogue categories 212 for similar dialogue impressions andcorresponding provider systems 103. In various embodiments, thecorrelation may be based at least in part on matching selected keywordsand/or expressions to identical and/or similar keywords and/orexpressions specified for certain dialogue categories 212 andcorresponding provider systems 103. The dialogue categories 212 mayinclude categorizations of concept, keyword, expression, and/or the likemapped to corresponding provider systems 103. Based at least in part onthe dialogue impression, the matching engine 238 may create a dialogueprofile for content items 176 and/or programming content 202 andcorresponding provider systems 103. The dialogue profile may be retainedin any suitable form, such as a file, a list, etc. The content compositesubsystem 111 may store dialogue profiles mapped to content items 176and/or programming content 202 and corresponding provider systems 103 inthe repository 222. Each dialogue profile may specify one or moreselected keywords and/or expressions as distinctive markings and/ordialogue categories 212 as characteristics of the content items 176and/or programming content 202 and corresponding provider systems 103.

To facilitated the matching, the matching engine 238 may performcorrelation based at least in part on the text recognition, imagerecognition, and/or audio recognition and text, image, and/or audiocategories 212. By way of example, the correlation rules 218 may includecorrelation criteria that could include keywords identified by any oneor combination of words, word stems, phrase, word groupings, and/or likekeyword information. The correlation criteria could include weightingsassigned to dialogue specifications. Hence, within each dialogue bucket,a keyword could be assigned a weight according to its significance. Moresignificant keywords could be assigned a greater weight than stop words.Such buckets could be implemented in any suitable manner, includinglists, tables, matrices, and/or the like. And within a bucket,characteristics and/or keywords could be organized in a rank orderand/or any hierarchical structure according to weight. Some embodimentsmay have characteristics and/or keywords organized according to decisiontree, with contingencies so that only certain combinations ofcharacteristics and/or keywords may be considered. For example, certaincharacteristics and/or keywords could only have significance if used inconjunction with other characteristics and/or keywords, and/or not inconjunction with others. Similar correlation criteria may be used foracoustic impressions.

In some embodiments, the matching engine 238 may employ a scoring systemto quantify correlations with a numerical expression, for example, amatch score, with higher scores being assigned to higher correlations.Higher scores may be assigned for greater extents of matching. By way ofexample with respect to dialogue, a match of three dialogue cues ordialogue categories may be assigned a higher score than a match of onlyone top dialogue cue and/or dialogue category. As another example, anexact match of words, expressions, and/or dialogue categories may beassigned a higher score than a complimentary match (e.g., where absentan exact match, a word, expression, and/or dialogue category is definedas a complement to another a word, expression, and/or dialoguecategory).

Some embodiments may sample only portions of content items 176 and/orprogramming content 202 to derive the characteristics of the contentitems 176 and/or programming content 202. For example, the sampling maybe relegated to only the last X seconds of audio and/or video of acontent item 176. As another example, the sampling may be relegated toonly I-frames of a content item 176. Accordingly, the sampling mayfacilitate composite 180 creation and presentation so that the composite180 may be dynamically inserted in the content stream or otherwiseprovisioned within a short time and with minimized processing resources.

In some embodiments, with a system 103 identified based at least in parton identified the metadata and/or characteristics, the matching engine238 may and perform a search of the repositories 222 for one or morecontent items 177 that have metadata that maps to provider systems 103.In such cases where the repositories 222 are searched for one or morecontent items 177, the one or more content items 177 may have beenpreviously transmitted to the subsystem 111 directly from one or moreservice provider systems 103-1 or indirectly from one or more serviceprovider systems 103-1 by way of the service provider system 102-1. Suchcontent items 177 may include temporal identification indicating a timefor which content corresponding to the content item 177 remains ineffect. For example, each content item 177 may comprise content forwhich an expiration is specified (e.g., with one or more of time, date,duration, etc.). Accordingly, the matching engine 238 may perform astaleness check by examining the metadata of any such content items 177retrieved from the repositories 222 to determine whether the contentitems 177 have expired with respect to the current time and date.

Additionally or alternatively, the subsystem 111 may pull one or morecontent items 177 from the service provider system 102-1 and/or one ormore service provider systems 103-1. In so doing, the subsystem 111 maytransmit at least a portion of the metadata of the programming content202 to the service provider system 102-1 and/or one or more serviceprovider systems 103-1 in order to facilitate matching the extractedmetadata with one or more content items 177 with respect to one or moreof event identification, event category identification, and/or temporalidentification. Consequently, the service provider system 102-1 and/orone or more service provider systems 103-1 may transmit one or morematching content items 177 to the subsystem 111. Additionally oralternatively, in some embodiments, the subsystem 111 first obtain oneor more matching adaptable content items 176, then read metadata fromthe one or more matching adaptable content items 176, and transmit atleast a portion of the metadata to the service provider system 102-1and/or one or more service provider systems 103-1 in order to facilitatematching the metadata with one or more content items 177.

Some embodiments may include the subsystem 111 being configured toperform a search of the repositories 222 for one or more content items177 that have metadata matching the metadata of one or both of adaptablecontent items 176 and the segment of programming content 202 in additionto pulling one or more content items 177 from the service providersystems 102-1 and/or 103-1. For example, the subsystem 111 may firstperform a search of the repositories 222 for any matching content items177 and then only pull one or more content items 177 from the serviceprovider systems 102-1 and/or 103-1 when no non-expired content items177 are found in the search of the repositories 222 that match themetadata with a sufficient match score that indicates a level ofcorrelation satisfying a correlation threshold. In some embodiments, thesubsystem 111 may first attempt to pull one or more content items 177from the service provider system 102-1 and, in the case that no matchingnon-expired content items 177 are successfully pulled from the serviceprovider system 102-1 within a time threshold after the initial request,then pull one or more content items 177 from one or more serviceprovider systems 103-1. Likewise, the subsystem 111 may attempt to pullone or more content items 177 from a first set of one or more serviceprovider systems 103-1 and, in the case that no matching content items177 are successfully pulled from the first set of one or more serviceprovider system 103-1 within a time threshold after the initial request,then pull one or more content items 177 from a second set of one or moreservice provider systems 103-1. In some embodiments, the subsystem 111may implement all three of the aforementioned stages of pulling contentitems 177 from the service provider system 102-1, then the first set ofone or more service provider systems 103-1, and then a second set of oneor more service provider systems 103-1. In such embodiments, the timethresholds may be different for each stage, e.g., decreasing orincreasing in time. Additionally, the size of the selected sets ofservice provider systems 103-1 may be different for each stage, e.g.,decreasing or increasing in size. Accordingly, various embodiments mayfacilitate matching of particularized content items 177 to segments ofprogramming content 202 and/or adaptable content items 176 based atleast in part on metadata.

The matching engine 238 may take the identified characteristics of agiven segment of programming content 202 as cues to retrieve one or morecorresponding adaptable content items 176 and one or more content items177 for the content splicing engine 242 to output one or more contentcomposites 180 for presentation after the segment of programming content202 (e.g., at a commercial break) or during the segment of programmingcontent 202 (e.g., as an overlay, in a side window/frame, etc.) to matchthe identified characteristics of the programming content 202. Accordingto some embodiments, the matching engine 238 may employ one or moreartificial intelligence algorithms to identify characteristic patternsin the programming content 202 which may correspond to triggers forintra-event creation of composites 180 for presentation at an upcomingbreak or during the event.

As one example case out of many possibilities, say a state change in atelevised event of programming content 202 occurs (e.g., a score change,one team or competitor leading or falling behind, a takedown, a foul,etc.) and/or temporal progression marker (e.g., the fourth quarter, thesecond round, etc.) is reached. The subsystem 111 may detect the statechange and, in response, initiate creation of composites 180 forpresentation at an upcoming break or during the event. Accordingly, thedialogue may trigger composite 180 creation and presentation so that thecomposite 180 may be dynamically inserted in the content stream within ashort time (e.g., substantially, seconds, within a minute, withenvironments, etc.) after the state change. In various embodiments, thesubsystem 111 may detect the state change the way of keyword recognitionof a dialogue from an announcer (e.g., detecting words such astouchdown, goal, takedown, foul, minutes on the clock, etc.), suddenchanges in crowd noise, and/or image recognition (e.g., detectinggraphics displayed with a televised event such as a scoreboard,streaming tickers or overlays typically positioned and/or scrollingacross a portion of the display area, etc.).

To facilitate audio recognition, the content composite subsystem 111(e.g., by way of the matching engine 238) may be configured to detectone or more audio characteristics in adaptable programming content 202.In some embodiments, the matching engine 238 may include an audioanalyzer and handling module to facilitate that detection. By way ofexample, the matching engine 238 may detect one or more audiocharacteristics in programming content 202 by way of any one orcombination of analyzing closed captioning information, analyzing audio,applying voice recognition, acoustic spectrum analysis, comparison toacoustic profiles for types of events, and/or the like.

Some embodiments of the content composite subsystem 111 may storeacoustic profiles mapped to types of programming content 202 andcorresponding state changes in the processed content repository. Eachacoustic profile may specify one or more selected values of acousticmetrics as distinctive markings and/or acoustic categories 214 ascharacteristics of the types of programming content 202 andcorresponding state changes. In various embodiments, the matching engine238 may analyze the audio tracks metadata track, and/or close captioninginformation to identify dialogue as distinctive markings of statechanges. The matching engine 238 may identify one or more keywordsand/or expressions in the dialogue as a dialogue impression for thepurposes of characterizing the sample and identifying state changes. Thematching engine 238 may compile the keywords and/or expressions of thesample and retain the keywords and/or expressions and correspondingstate changes in the dialogue impression. In some cases, the matchingengine 238 may correlate the dialogue impression to one or more dialoguecategories 212 for similar dialogue impressions and corresponding statechanges. In various embodiments, the correlation may be based at leastin part on matching selected keywords and/or expressions to identicaland/or similar keywords and/or expressions specified for certaindialogue categories 212. The dialogue categories 212 may includecategorizations of concept, keyword, expression, and/or the like mappedto corresponding state changes.

Based at least in part on the dialogue impression, the matching engine238 may create a dialogue profile for a given segment and/or type ofprogramming content 202 and corresponding state changes. The dialogueprofile may be retained in any suitable form, such as a file, a list,etc. The content composite subsystem 111 may store dialogue profilesmapped to segments of programming content 202 and corresponding statechanges in the processed content repository. Each dialogue profile mayspecify one or more selected keywords and/or expressions as distinctivemarkings and/or dialogue categories 212 as characteristics of thesegment and/or type of programming content 202 and corresponding statechanges.

To facilitate image recognition, some embodiments of the contentcomposite subsystem 111 (e.g., by way of the matching engine 238) may beconfigured to detect one or more color characteristics in programmingcontent 202, which may include any suitable color characteristic thatmay be measured recognize alphanumerical images corresponding to suchstate changes. In some embodiments, the matching engine 238 may includean image analyzer and handling module to facilitate that detection. Byway of example, the matching engine 238 may detect one or more colorcharacteristics in programming content 202 by way of any one orcombination of pixel identification, detecting color component values,detecting color codes, and/or the like. Some embodiments may sample onlyportions of images, such as only part of a frame (which could, forexample, be specified by identification of pixel coordinates and rangesthereof to define areas of interest). Some embodiments may sample only acentral portion of a frame. Other embodiments may only sample anon-central portion, such as a border portion of a frame. Someembodiments may start with one portion of a frame and only sample tountil a state change is recognized.

The matching engine 238 may perform correlation based at least in parton correlation rules 218. The matching engine 238 may correlate segmentsof programming content 202 to state changes based at least in part onone or a combination of such characteristics. In some embodiments, thecorrelation may be based at least in part on the color profiles,acoustic profiles, and/or dialogue profiles of types and/or segments ofprogramming content 202. In some embodiments, the correlation may bebased at least in part on color, dialogue, and/or acoustic categories212.

The correlation rules 218 may include correlation criteria that couldinclude keywords identified by any one or combination of words, wordstems, phrase, word groupings, and/or like keyword information. Thecorrelation criteria could include weightings assigned to dialoguespecifications. Hence, within each dialogue bucket, a keyword could beassigned a weight according to its significance. By way of example, moresignificant keywords could be assigned a greater weight than stop words.Such buckets could be implemented in any suitable manner, includinglists, tables, matrices, and/or the like. And within a bucket,characteristics and/or keywords could be organized in a rank orderand/or any hierarchical structure according to weight. Some embodimentsmay have characteristics and/or keywords organized according to decisiontree, with contingencies so that only certain combinations ofcharacteristics and/or keywords may be considered. For example, certaincharacteristics and/or keywords could only have significance if used inconjunction with other keywords, and/or not in conjunction with others.

In some embodiments, the matching engine 238 may employ a scoring systemto quantify correlations with a numerical expression, for example, amatch score, with higher scores being assigned to higher correlations ofsegments of programming content 202 with state changes. Higher scoresmay be assigned for greater extents of matching. For example, a match ofthree dialogue cues or dialogue categories may be assigned a higherscore than a match of only one top dialogue cue and/or dialoguecategory. As another example, an exact match of words, expressions,and/or dialogue categories may be assigned a higher score than acomplimentary match (e.g., where absent an exact match, a word,expression, and/or dialogue category is defined as a complement toanother a word, expression, and/or dialogue category).

Further, the matching engine 238 may correlate the state changes tocorresponding adaptable content items 176 that the subsystem 111 hasstored locally to identify one or more of which that are particularizedto some extent to the types of state changes detected. In someembodiments, a set of adaptable content items 176 may be rankedaccording to correlation such that the strongest matching content item176 is ranked highest, and the ranked set may be selected for composite180 creation in the ranked order consequent to the matched segment ofprogramming content 202.

As disclosed herein, the subsystem 111 may determine that theprogramming content 202 corresponds to a televised content segment forwhich content composite 180 creation is an option. The subsystem 111 maydetect a location corresponding to the computing device 116 and thereceiver 111, and may determine from rules 218 whether the provisioningof content composites 180 of certain type are prohibited for thelocation. In the case where the subsystem 111 determines that there isno prohibition for the location, the subsystem 111 may determine a setof the rules 218 that govern timing restrictions and specifications,event type restrictions and specifications, place and mannerrestrictions and specifications, and types of adaptations of theadaptable content items 176 with the particularized content items 177 tocreate the composites 180. Various sets of rules 218 may provide forvarious types of adaptations of the adaptable content items 176 with theparticularized content items 177, and the subsystem 111 may determinewhich set of rules 218 apply to a given receiver 116 e and/or device 116as a function of the location of the receiver 116 e and/or device 116.

The place and manner restrictions and specifications of the rules 218may govern how and when composites are provisioned with respect toprogramming content 202 (e.g., as a commercial, as a real-time pop-up,as a real-time overlay, as an inset frame, a sliding and swipableoverlay, and/or like), which may be a function of the type of program,type of end user (e.g., demographics), history and patterns of userinput and selections, and/or the type of end-user device. For example, aset of rules 218 may specify that composites 180 may only be providedduring commercial breaks of program. Another set of rules 218 mayspecify that composites 180 may be provided as overlays, frames, and/orpop-ups during the televised event.

The rules 218 may require the user opt in and request particularizedcontent items 177. Accordingly, one or more user-selectable options maybe presented to the user via the receiver 116 e and/or device 116 toallow presentation of particularized content items 177 and/or options torequest particularized content items 177. Such user-selectable optionsmay be provided with composites 180 that are presented and duringcommercial breaks. Likewise, one or more user-selectable options may bepresented to the user via the receiver 116 e and/or device 116 toprohibit provisioning of particularized content items 177 and/or optionsto request particularized content items 177. Upon user selection of suchprohibition, the subsystem 111 may present alternative content items inlieu of particularized content items 177 and/or options to requestparticularized content items 177 during commercial breaks, such ascontent items 176 without particularized content items 177 and/oroptions to request particularized content items 177 and/or other contentitems. Consequent to a user opting in, prohibiting, and/or requestingparticularized content items 177, one or more communications may betransmitted to the system 102 to indicate the user selection(s). Thesystem 102 may update an account associated with the user and/orendpoint media device 116 accordingly, for example, with storage of userpreferences and credits/awards for user interactions with respect tocontent items 177.

In some embodiments, during a period of time when a content item 176 isbeing presented with a display of the endpoint device 116 (e.g., with acommercial), a user-selectable option may be presented to allow for userselection to request one or more particularized content items 177corresponding to the content item 176. In some embodiments, a requestfor one or more particularized content items 177 corresponding to thecontent item 176 may be made by way of one or more buttons on a remotecontrol associated with a television receiver 116 e that are configuredfor selection of a displayed user-selectable option or otherwise torequest a particularized content item 177 corresponding to the contentitem 176 being output by the television receiver 116 e for display. Inaddition or in alternative, one or more user-selectable options may beavailable during the time period for on-screen selection via a remotecontrol and a selection function (e.g., an on-screen selector such as ahovering indicator of any suitable form) and/or one or more buttons ontelevision receiver 116 e that allows a user to interact with televisionreceiver 116 e. In some embodiments, an indication of theuser-selectable option associated with a content item 176 representedon-screen could be presented to indicate that the user may select anoption to obtain further information.

Additionally or alternatively, in some embodiments, a selection of auser-selectable option and/or another type of request may be made orallyby user and recognized with a voice recognition device integrated withthe remote control or integrated with or otherwise communicativelycoupled to the endpoint media device 116. For example, the voicerecognition device may correspond to a smart speaker with virtualassistant. Thus, a user may user opt in and request particularizedcontent items 177 via voice recognition.

Particularized content items 177 may be provisioned to endpoint mediadevices 116 in one or a combination of ways. In some embodiments,particularized content items 177 may be integrated and displayed withthe adaptable content item 176. Such presentations of the particularizedcontent items 177 may be captured by an endpoint media device 116 viascanning with a scanner, taking a photo with a camera, saving ascreenshot, and/or the like. Additionally or alternatively,particularized content items 177 may transferred to an accountaccessible by the user of the endpoint media device 116. Additionally oralternatively, particularized content items 177 may be provisioned to adevice 116 by way of one or combination of application installed on thedevice 116, communications from a receiver 116 e, communications fromthe service provider system 102-1, a near field communication interface(e.g., contactless interface, Bluetooth, optical interface, etc.),wireless communications interfaces capable of communicating through acellular data network, or through Wi-Fi, such as with a wireless localarea network (WLAN), and/or the network 120. By way of example, acomposite 180 may be presented via a television receiver 116 e on adisplay 160 with one or more user-selectable options that allowredirection of composites 180 and/or particularized content items 177 toa secondary device 116 consequent to user selection. Further, one ormore user-selectable options may be presented to the user via thereceiver 116 e and/or device 116 to allow content composite 180 and/orparticularized content item 177 provisioning to a secondary device 116concurrently with the presentation of the televised content segment.

The user-selectable options presented with the composite 180 may allowfor taking actions, such as selecting one or more URLs, hyperlinks,click-to-call, and/or voice-activate-to-call options to one or moreplatforms, websites, and/or sites for further information, ordering,requesting, and/or accessing products and/or services. As disclosedherein, the user-selectable options may include one or more options totransition provisioning of one or more composites 180 to a secondarydevice 116 and/or to request notifications from the one or moreplatforms, websites, and/or sites be sent to the secondary device 116 sothat the secondary device 116 may be used to interact with the platformsand sites via an application installed on the secondary device 116. Inthat way, a user may interact with the one or more platforms and sitesvia the secondary device 116 while viewing the televised event on aprimary display associated with a receiver 116 e and/or primary device116.

According to one option, the secondary device 116 may receive the samecontent, including composites 180 as commercial segments and/oroverlays, being displayed on the display device 160 with simulcasting tothe secondary device 116 so that the secondary device 116 need onlydisplay the augmented content. According to another option, thetelevision receiver 116 e may provide the programming content to thedisplay device 160, and the secondary device 116 may receive theprogramming content and splice composites 180 into the content displayedwith the device 116. In various embodiments, the device 116 may receivethe composites 180 for splicing from the receiver 116 e, may receive thecomposites 180 for splicing from the service provider system 102-1,and/or may receive adaptable content items 176 and content items 177from the receiver 116 e and/or the system 102-1 in order that the device116 may create provision composites 180 therefrom. In some modes ofoperation, the television receiver 116 e may present alternative contentitems with the display device 160 in lieu of content composites 180,while the content composites may be shunted to the secondary device 116.Thus, the secondary device 116 may receive composites 180 that wouldotherwise be displayed on the display device 160. In some embodimentsand options, the receiver 116 e may not cast to the televised event, butthe device 116 may present composites 180 without the televised event.For example, according to some options, an application for the device116 may be downloaded, installed, and initiated to facilitate contentprovisioning on the device 116 and interaction with one or a combinationof the receiver 116 e, system 102-1, and/or one or more systems 103-1.Accordingly, various embodiments may provide various user-selectableoptions for transitioning from just viewing a televised event todisplaying and interacting with composites 180 and service providersystems 103-1 via a secondary device 116, while a televised eventcorresponding to the composites 180 is being displayed on another devicesuch as display device 160 the receiver 116 e.

In accordance with the geo-specific rules 218, the subsystem 111 maydetect a trigger for composite 180 creation that corresponds to a needfor presentation of one or more composites 180 during an upcoming breakin the programming content 202 corresponding to an event and/or duringthe presentation programming content 202 corresponding to the event.Detecting a trigger may include determining timing specifications of theevent and a temporal progression in the event (e.g., the fourth quarter,the second round, etc.), detecting a state change in the event (e.g., ascore change, one team or competitor leading or falling behind, etc.),and/or the like. In various embodiments, a trigger may correspond to thematching engine 238 processing the metadata associated with programmingcontent 202 and/or processing the audio and/or video content of theprogramming content 202 to identify an upcoming break in the programmingcontent 202. This may include processing the metadata to identify flagsor other indicators of the timing specifications of a commercial break.Additionally or alternatively, this may include the audio recognition,video recognition, and/or closed captioning recognition disclosed aboveto identify timing specifications, temporal progression, and/or statechanges as triggers for composite 180 creation.

When the subsystem 111 detects a trigger, the subsystem 111 may receiveand/or select from one or more adaptable content items 176 provided bythe system 102-1 as matching particular segments of the televisedcontent segment and one or more content items 177 to generate and outputfor display, utilizing the content splicing engine 242, one or morecomposites 180 to display as a commercial during a programming breakafter a particular segment or as an intra-program overlay, frame,window, pop-up, and/or the like presented concurrently with thetelevised event. For example, a content item 177 may include text, oneor more images, links, URLs, buttons, other user interface elements,and/or the like which the content splicing engine 242 may aggregate,process, format, crop, rescale, and/or otherwise prepare and include incomposites 180 for insertion into the content stream for output with theprogramming content 202 and/or during breaks of the programming content202. Thus, the content splicing engine 242 may identify content portionsfor augmentation by processing the content item 177, reading the contentitem 177 or certain portions thereof, and determine portions foraugmentation in video segments. In some embodiments, portions of imagesand/or frames of the adaptable content item 176 may be overwritten withcaptured content from the content item 177.

The content splicing engine 242 may identify one or more portions ofadaptable content item 176 to be augmented based at least in part oncontent item 177, which may, in various embodiments, be identified byimage analysis and/or analysis of tag data that defines one or moreareas within frames that correspond to certain portions representedwithin the frames for augmentation. As disclosed herein, such tag datacould define the area of interest in any suitable way in variousembodiments which could be by way of any one or combination of mattes,masks, pixel identification (which could, for example, includeidentification of pixel coordinates and ranges thereof to define areasof interest), pixel color component characteristics such as colorcomponent values, overlays, and/or the like, allowing for correlation tothe area to be augmented in any suitable way. In some embodiments, aprocessor (such as a main processor, a core processor, digital signalprocessor, and/or like) may take a definition of the augmentation areawith respect to one or more reference frames and may performauto-correlation of related images in a video stream to identify/definethe augmentation areas in other frames of the video sequence thatrepresent the portion to be augmented. Image characteristics (e.g.,color, brightness, etc.) of the area of interest with respect to thereference frame(s) could be measured, quantified, and/or otherwiseidentified, and matched with measured image characteristics of the otherframes to define the area to be augmented in multiple frames in animage-changing sequence. Accordingly, certain embodiments may allow forhandling the complexity of multiple on-screen options by differentiatingparticular areas in dynamic, image-changing content.

As illustrated by FIG. 3 , the content processing subsystem 111 mayinclude a learning engine 239 that may be an analysis engine thatemploys machine learning. The learning engine 239 may further employdeep learning. Accordingly, the learning engine 239 may facilitatemachine learning or, more specifically, deep learning, to facilitatecreation, development, and/or use of viewer pattern data 216. Asdisclosed herein, the subsystem 111 may determine a program, show, etc.that the viewer actually is viewing, is about to view (e.g., thetelevised event is scheduled to play on the channel that the viewer iscurrently viewing), or is likely to view as determined with the learningengine 239. The subsystem 111 may push information indicating thetelevised event to one or more service provider systems 102-1 and/or103-1. In some embodiments, the service provider system 102-1 may selectone or more adaptable content items 176 matching the televised contentsegment for transfer to the subsystem 111 which, as disclosed herein,may be a part of the content provider system 102 and/or may be part ofthe receiver 116 e and/or devices 116. The subsystem 111 may select fromthe one or more adaptable content items 176 as matching particularsegments of the televised segment and, utilizing a content splicingengine 242 in some embodiments, may output one or more correspondingcontent composites 180 for display after the particular segments and/orsimultaneously with the particular segments.

In various embodiments, one or more of the service provider systems102-1, 103-1 may select one or more particularized content items 177matching the televised event for transfer to the subsystem 111. In someembodiments, one or more of the service provider systems 102-1, 103-1may select a set of one or more particularized content items 177 fortransfer (e.g., based on recency of information updates corresponding tothe content items 177) for transfer to the subsystem 111, and thesubsystem 111 may determine which content items 177 from the set matchthe televised content segment. As disclosed above, the content compositesubsystem 111 may include a matching engine 238 that may include logicto implement and/or otherwise facilitate any taxonomy, classification,categorization, correlation, mapping, qualification, scoring,organization, and/or the like features disclosed herein.

FIG. 5 illustrates certain aspects of the AI-based subsystem data flow500, in accordance with various embodiments of the present disclosure.The content processing subsystem 111 may be configured to gatherobservation data 229, which may be specific to one or more particularidentified users and/or may be generally related to particular endpointmedia devices 116. The observation data 229 may be gathered from one ormore devices 116, aggregated, consolidated, and transformed into viewerpattern profiles that include personalized pattern data 216.

In embodiments where the learning engine 239 is included in an endpointmedia device 116, the device 116 may be a self-observer that mayadditionally gather additional observation data 229. In variousembodiments, the data from the one or more devices 116 may be retrievedand/or received by the content processing subsystem 111 via one or moredata acquisition interfaces, which may include interfaces of the contentprocessing subsystem 111, the one or more endpoint media devices 116,and/or the like—through network(s) 120 in various embodiments, throughany suitable means for direct communication, and/or through any othersuitable means of transferring data. According to various embodimentswhere the subsystem 111 is included in a service provider system 102-1,observation data 229 may be actively gathered and/or pulled from the oneor more endpoint media devices 116.

As disclosed herein, in various embodiments, the one or more dataacquisition interfaces may include one or more APIs that defineprotocols and routines for interfacing with the one or more endpointmedia devices 116 and which may specify API calls to/from one or moreendpoint media devices 116. In various embodiments, the APIs may includea plug-in to integrate with an application of one or more endpoint mediadevices 116. The API translation profiles may translate the protocolsand routines of the data source component and/or system to integrate atleast temporarily with the system and allow one-way communication to thesystem 102-1 and/or two-way communication with system 102-1 in variousembodiments by way of API calls.

Some embodiments of the subsystem 111 may aggregate observation data 229to derive device identification data 504, device operations 506,temporal data 508, and/or contextual data 510. The device identificationdata 504 may include any suitable data for identifying and trackingparticular receivers 116 e and devices 116; associated accounts,subscribers, and viewers; and/or the like disclosed herein. The deviceoperations data 506 may include any suitable data for identifying andtracking device operations and interactions as those disclosed herein.The contextual data 510 may include metrics and patterns of viewerinteractions/responses pursuant to provisioning of content composites180 and service provider system 103-1 biasing 181. For example, viewerresponses to content composites 180 provisioning may include indicationsof whether the viewer selected a user-selectable options provided withcomposites 180, the types of such selections, and/or types of consequentinteractions with service provider systems 103-1. For example, themetrics and patterns may take into account whether the viewer opted outof content composites 180, whether the viewer selected links ofcomposites 180 to interact with the platforms and sites of one or moreservice provider systems 103-1, whether the viewer selected options toredirect content composites 180 and/or notifications from serviceprovider systems 103-1 to a secondary device 116, which options and/orservice provider systems 103-1 the viewer selected, whether the viewerused redeemable content items, QR codes, etc. and the types of theviewer's orders, purchases, and other interactions with service providersystems 103-1, and/or the like. The temporal data 508 may includemetrics such as any information to facilitate detection, recognition,and differentiation of one or combination of temporal factors correlatedor which the content processing subsystem 111 correlates to otherobservation data 229 such as device identification data 504, contextualdata 510, and/or the like. For example, the temporal data 508 mayinclude time of day information, time of week information, time of yearinformation, holiday information, etc. when the viewer made selections,orders, purchases, redemptions of redeemable content items, offers,promo codes, etc.; and/or the like.

The learning engine 239 may map one or a combination of the variousextra-composite metrics of the observation data 229 to the metrics ofthe particular composites 180 provided to a particular viewer. Based atleast in part on taking into account such observation data 229 as partof a feedback loop, the learning engine 239 may employ an ongoinglearning mode to develop personalized pattern data 216 for particularviewers or content receivers/devices generally, and to confirm, correct,and/or refine determinations made for personalized pattern data 216 forparticular viewers or content receivers/devices generally. The contentprocessing subsystem 111 may be configured to employ machine learning toprocess the observation data 229 and the content items 180 and to deriveand develop the personalized pattern data 216. The content processingsubsystem 111 may be configured to employ deep learning to process theobservation data 229 and the content items 180 and to derive and developthe personalized pattern data 216. The learning engine 239 may beconfigured to perform any one or combination of features directed tomatching or otherwise correlating the observation data 229—such as thedevice identification data 504, the device operation identification data506, the temporal data 508, the contextual data 510, descriptiveinformation of the content items 180, and/or the like—with intra-contentmetrics of the content items 180. The learning engine 239 may includelogic to implement and/or otherwise facilitate any taxonomy,classification, categorization, correlation, mapping, qualification,scoring, organization, and/or the like features disclosed herein. Insome embodiments, the learning engine 239 may include the matchingengine 238.

The learning engine 239 may include a reasoning module to make logicalinferences from a set of the detected and differentiated data to inferone or more patterns of activity for particular viewers and/orreceivers/devices generally. A pattern-based reasoner could be employedto use various statistical techniques in analyzing the data in order toinfer personalized pattern data 216 from the observation data 229. Atransitive reasoner may be employed to infer relationships from a set ofrelationships related to the observation data 229. In variousembodiments, the system automatically establishes and develops thepersonalized pattern data 216. However, the personalized pattern data216 may be set up and/or tailored by users. With various embodiments,the personalized pattern data 216 may be automatically established anddeveloped by the system.

The feedback could be used for training the system to heuristicallyadapt conclusions, profiles, correlations, attributes, triggers,patterns, and/or the like to learn particular viewers and adapt contentcomposite 180 provisioning to particular viewers, which may includerequesting, searching for, and/or selecting particular types ofadaptable content items 176 and/or content items 177 (e.g., which may bebased at least in part on the metadata features disclosed herein) forcontent composite 180 creation. For example, the learning engine 239 maylearn that a particular viewer tends to interact with content composites180 that are directed to only certain types of products and/or services.Such interaction object type differentiation may be on the macro level,such as recognizing that a viewer tends to interact more with composites180 directed to certain types of products, services, and/or offers andnot other types of products, services, and/or offers. Additionally, thelearning engine 239 may learn that a particular viewer tends to interactwith particular content composites 180 and/or types only after theparticular content composite 180 and/or type has been presented to theviewer X number of times. Accordingly, the subsystem 111 may biascontent composites 180 provisioning toward the types of products and/orservices that tend to elicit viewer interaction, and decrease composite180 provisioning frequency or cease provisioning for other types.

Further, the learning engine 239 may learn that a particular viewertends to interact with content composites 180 that are directed to onlytypes of products, services, and/or offers within a particular category.Accordingly, the subsystem 111 may likewise bias content composites 180provisioning toward such types of products, services, and/or offers anddecrease or cease provisioning with respect to other types. Further,viewer interaction differentiation may be on the micro level, such asrecognizing that a viewer tends to interact more with composites 180directed to certain types of offers, redeemable content items,promotions, etc. Accordingly, the subsystem 111 may likewise biascontent composites 180 provisioning toward such types and decrease orcease provisioning with respect to other types.

Further, in situations where a pattern of more micro-level interactionsdetected for particular viewer, the subsystem 111 biasing 181 ofcomposites 180 may include serial provisioning of composites 180 in aserial drill-down manner such that the first composites 180 provisionedmay be directed to a more macro-level outcome and one or more composites180 subsequently provisioned may be directed to more micro-leveloutcomes in accordance with the detected pattern. Thus, disclosedembodiments may provide for serial matching of composites 180 withrespect to one another in order to provision the composites 180 with atrend that matches the detected pattern. As one possible example out ofmany, when viewer interaction with macro-level composite 180 directed toa macro-level product and/or service (e.g., a vehicle) results in amacro-level outcome by the viewer (e.g., an purchase, order, etc. of thevehicle), the biasing 181 may include ceasing provisioning of repeatsand variations of the macro-level composite 180 to the viewer and mayfurther include subsequently provisioning more micro-level contentcomposites 180 directed to more micro-level outcomes (e.g.,complementary products/services, such as warranties, accessories, etc.).

As part of such learning and adaptation processes, the subsystem 111 maybias composite 180 provisioning toward use of content items 177 fromcertain service provider systems 103-1 over other service providersystems 103-1. The learning engine 239 may perform correlation based atleast in part on correlation rules that govern correlation of thepersonalized pattern data 216 to content items 177 and correspondingsources of the content items 177 based at least in part on metrics andavailabilities of the content items 177 from the particular sourcesystems 103-1. In various embodiments, the correlation may be based atleast in part on the profiles of the service provider systems 103-1. Inanalyzing content items 177 received for particular service providersystems 103-1, the subsystem 111 may identify patterns of types ofcontent items 177 and mapped which types of content items 177 areprovided by which service provider systems 103-1. Based at least in parton such mapping, the subsystem 111 may prioritize use of content items177 from particular service provider systems 103-1 where the contentitems 177 are of viewers directed to the types of interactions thatmatch interaction patterns of a particular viewer.

Further, such biasing 181 of content items 177 from one or more systems103-1 may include prioritizing attempts to pull content items 177 fromone or more systems 103-1 based at least in part on speed andresponsiveness of systems 103-1. The subsystem 111 may aggregate anddevelop service provider system metrics based at least in part on thespeed and responsiveness of systems 103-1 demonstrated with previousattempts to pull one or more content items 177 within one or morehistorical time windows (e.g., previous day, five days, week, month,year, etc.). The service provider systems 103-1 may be scored based atleast in part on the metrics, and an individual scores may be utilizedin identifying preferential biasing 181 an attempt to pull content items177 from respective systems 103-1. For example, the subsystem 111 mayattempt to pull one or more content items 177 from a first set of one ormore service provider systems 103-1 that has the highest score based onprevious performance with respect to speed, timeliness, and a matchingcontent items 177. In the case that no matching content items 177 aresuccessfully pulled from the first set of one or more service providersystem 103-1 within a time threshold after the initial request, then thesubsystem 111 may pull one or more content items 177 from a second setof one or more service provider systems 103-1 that have lower scores.Such serial stages attempt to pull content items 177 may be continuedwith additional sets of lower scored systems 103-1. The time thresholdsmay be different for each stage (e.g., decreasing or increasing in time)and the size of the selected sets of service provider systems 103-1 maybe different for each stage (e.g., decreasing or increasing in size). Incases where multiple such stages are necessary, the subsystem 111 maysubsequently adjust metrics for the systems 103-1, taking into accountthe performance demonstrated with such stages of attempt to pull objects177.

In some embodiments, the device identification data 504 may includeendpoint media device 116 mappings that logically associate endpointmedia devices 116 that are linked based on one or more commonalities.For example, the learning engine 239 may learn that a set of endpointmedia devices 116 are mappable to a particular household, to aparticular company, and/or the like. Consequently, the learning engine239 may learn observation data 229 may be differentiated not only onindividual bases (e.g., with respect to individual devices 116,accounts, user profiles, etc.), but also on collective bases (i.e., withrespect to sets of devices 116, accounts, user profiles, etc.). Further,the adapting and provisioning of content items 176, 177, and/orcomposites 180 may be based at least in part on the learning of suchlogical associations. For example, each endpoint media device 116 of aset of endpoint media devices 116 mapped to a particular household maybe provisioned with a common set of content items 176, 177, and/orcomposites 180, regardless of the previous content items 176, 177,and/or composites 180 provisioned to each endpoint media device 116 inthe set. In some embodiments, the common set of content items 176, 177,and/or composites 180 may be related to, but different from, one or moreprevious content items 176, 177, and/or composites 180 provisioned toone or more endpoint media devices 116 in the set for a sharedexperience. Likewise, the common set of content items 176, 177, and/orcomposites 180 may be related to one or more interactions one or moreendpoint media device 116 in the set with one or more previous contentitems 176, 177, and/or composites 180 provisioned to one or more of theendpoint media devices 116. For example, a user selection of a hyperlinkin a content composite 180 may trigger a related content item 177 beingprovisioned to all endpoint media devices 116 in the set. In likemanner, any of the features disclosed herein with respect to adapting,provisioning, and biasing content items 176, 177, and/or composites 180may be based at least in part on such linking of endpoint media devices116.

FIG. 6 illustrates a receiver 600 that makes use of, interacts with,and/or at least partially includes the subsystem 111, in accordance withdisclosed embodiments of the present disclosure. Certain embodiments ofthe receiver 600 may include set top boxes (STBs), television receivers,and over-the-top receivers. In some embodiments, the receiver 600 maycorrespond to the television receiver 116 e. In various embodiments, inaddition to being in the form of a STB, a receiver may be incorporatedas part of another device, such as a television or other form of displaydevice, such as a computer, smartphone, tablet, or other handheldportable electronic device. For example, a television may have anintegrated receiver (which does not involve an external STB beingcoupled with the television). One or a combination of the contentharvesting engine 236-1, the content matching engine 238-1, learningengine 239-1, content composite build engine 240-1, and/or contentsplicing engine 242-1 may be provided in conjunction with contentharvesting module 236-2, the content matching module 238-2, contentaugmentation module 240-2, and/or content splicing module 242-2 toimplement various functionalities of the content composite subsystem 111into the receiver 600.

The receiver 600 may represent receiver 116 e of FIG. 1 and may be inthe form of a STB that communicates with a display device such as atelevision. The receiver 600 may be incorporated as part of atelevision, such as the display device 160 of FIG. 1 or television 600of FIG. 6 , etc. The receiver 600 may include: processors 610 (which mayinclude control processor 610-1, tuning management processor 610-2, andpossibly additional processors), tuners 615, network interface 620,non-transitory computer-readable storage medium 625, electronicprogramming guide (EPG) database 630, networking information table (NIT)640, digital video recorder (DVR) database 645, on-demand programming627, content store 222-3, user interface 650, decryption device 660,decoder module 633, interface 635, and/or descrambling engine 665. Inother embodiments of receiver 600, fewer or greater numbers ofcomponents may be present. It should be understood that the variouscomponents of receiver 600 may be implemented using hardware, firmware,software, and/or some combination thereof. Functionality of componentsmay be combined; for example, functions of descrambling engine 665 maybe performed by tuning management processor 610-2. Further,functionality of components may be spread among additional components;for example, PID filters 655 may be handled by separate hardware fromprogram map table 657. The receiver 600 may be in data communicationwith service providers, such as by way of network interface 620.

The processors 610 may include those configured to perform processessuch as tuning to a particular channel, accessing and displaying EPGinformation from the EPG database 630, and/or receiving and processinginput from a user. For example, the processors 610 may include one ormore processors dedicated to decoding video signals from a particularformat, such as MPEG, for output and display on a television and forperforming decryption, if required. It should be understood that thefunctions performed by various modules of FIG. 6 may be performed usingone or more processors. As such, for example, functions of thedescrambling engine 665 may be performed by the control processor 610-1.

The control processor 610-1 may communicate with tuning managementprocessor 610-2. The control processor 610-1 may control the recordingof television channels based at least in part on timers stored in theDVR database 645. The control processor 610-1 may also provide commandsto the tuning management processor 610-2 when recording of a televisionchannel is to cease. In addition to providing commands relating to therecording of television channels, the control processor 610-1 mayprovide commands to the tuning management processor 610-2 that indicatetelevision channels to be output to the decoder module 633 for output toa display device. The control processor 610-1 may also communicate withthe network interface 620 and the user interface 650. The controlprocessor 610-1 may handle incoming data from network interface 620,including network transport streams received from over-the-top serviceproviders. The control processor 610-1 may handle incoming data from thenetwork interface 620, including network transport streams received fromthe user interface 650, which may include user input received by way ofone or more human interface devices. Additionally, the control processor610-1 may be configured to output data via the network interface 620.The control processor 610 may also be configured to perform image andaudio analysis, such as to identify contextual aspects present in anaudio or video stream, for example facial recognition or voicerecognition.

The tuners 615 may include one or more tuners used to tune totransponders that include broadcasts of one or more television channelsthat may be received from a satellite or cable system, for example. Inthe illustrated embodiment of receiver 600, three tuners are present(tuner 615-1, tuner 615-2, and tuner 615-3). In other embodiments, twoor more than three tuners may be present, such as four, six, or eighttuners. Each tuner contained in the tuners 615 may be capable ofreceiving and processing a single transponder stream from a satellitetransponder at a given time, for example. As such, a single tuner maytune to a single transponder stream at a given time. If the tuners 615include multiple tuners, one tuner may be used to tune to a televisionchannel on a first transponder stream for display using a television,while another tuner may be used to tune to a television channel on asecond transponder for recording and viewing at some other time. Ifmultiple television channels transmitted on the same transponder streamare desired, a single tuner of the tuners 615 may be used to receive thesignal containing the multiple television channels for presentationand/or recording. The tuners 615 may receive commands from the tuningmanagement processor 610-2. Such commands may instruct the tuners 615which frequencies are to be used for tuning.

The network interface 620 may be used to communicate via an alternatecommunication channel with a television service provider, if suchcommunication channel is available. The primary communication channelmay be via satellite (which may be unidirectional to the receiver 600)and the alternate communication channel (which may be bidirectional) maybe via a network, such as the Internet. Referring back to FIG. 1 ,receiver 116 e may be able to communicate with content provider system102-1 via a network 120, such as the Internet. This communication may bebidirectional: data may be transmitted from the receiver 116 e to thecontent provider system 102-1 and from the content provider system 102-1to the receiver 116 e. Referring back to FIG. 6 , the network interface620 may be configured to communicate via one or more networks, such asthe Internet, to communicate with content provider system 102-1 of FIG.1 . Other information may be transmitted and/or received via the networkinterface 620 such as adaptable content items 176, content items 177,metadata, and/or the like as disclosed herein.

The storage medium 625 may represent one or more non-transitorycomputer-readable storage mediums. The storage medium 625 may includememory and/or a hard drive. The storage medium 625 may be used to storeinformation received from one or more satellites and/or informationreceived via the network interface 620. The storage medium 625 may storeinformation related to the EPG database 630, augmentation module 632 andrelated preferences, other non-video/audio data 631, DVR database 645,commercial database 646, and/or on-demand programming 627. Recordedtelevision programs may be stored using the storage medium 625 as partof the DVR database 645. The storage medium 625 may be partitioned orotherwise divided (such as into folders) such that predefined amounts ofthe storage medium 625 are devoted to storage of television programsrecorded due to user-defined timers and stored television programsrecorded due to provider-defined timers.

The EPG database 630 may store information related to televisionchannels and the timing of programs appearing on such televisionchannels. The EPG database 630 may be stored using the storage medium625, which may be a hard drive. Information from the EPG database 630may be used to inform users of what television channels or programs arepopular and/or provide recommendations to the user. Information from theEPG database 630 may provide the user with a visual interface displayedby a television that allows a user to browse and select televisionchannels and/or television programs for viewing and/or recording.Information used to populate the EPG database 630 may be received viathe network interface 620 and/or via satellites, such as the satellite106 of FIG. 1 via the tuners 615. For instance, updates to the EPGdatabase 630 may be received periodically via satellite. The EPGdatabase 630 may serve as an interface for a user to control DVRfunctions of the receiver 600, and/or to enable viewing and/or recordingof multiple television channels simultaneously. Information from EPGdatabase 630 may be output as a video stream to a display device. Aparticular user may issue commands indicating that an EPG interface bepresented. A user issuing a command that an EPG be displayed mayconstitute a change command. In some embodiments, content composites 180may be created and presented in conjunction with the EPG. For example,content composites 180 could pertain to televised events indicated inthe EPG. Hence, content composite 180 features may extend to EPG viewsin some embodiments.

The network information table (NIT) 640 may store information used bythe receiver 600 to access various television channels. The NIT 640 maybe stored locally by a processor, such as the tuning managementprocessor 610-2 and/or by the storage medium 625. Information used topopulate the NIT 640 may be received via satellite (or cable) throughthe tuners 615, may be received via the network interface 620, such asfrom the television service provider. As such, information present inthe NIT 640 may be periodically updated. In some embodiments, NIT 640may be locally-stored by the receiver 600 using the storage medium 625.Generally, the NIT 640 may store information about a service providernetwork, such as a satellite-based service provider network. Informationthat may be present in the NIT 640 may include: television channelnumbers, satellite identifiers (which may be used to ensure differentsatellites are tuned to for reception of desired television channels),frequency identifiers, transponder identifiers for various televisionchannels, antenna identifiers (which may be used to ensure differentantennas are tuned to for reception of desired television channels),radio frequency identifiers, and/or subchannel identifiers for varioustelevision channels. In some embodiments, the NIT 640 may containadditional data or additional tables may be stored by the receiver. Forexample, while specific audio PIDs and video PIDs may not be present inthe NIT 640, a channel identifier may be present within the NIT 640which may be used to look up the audio PIDs and video PIDs in anothertable, such as a program map table (PMT). In some embodiments, a PIDassociated with the data for the PMT is indicated in a separate table,program association table (PAT), which is not illustrated in FIG. 6 . APAT may be stored by the receiver in a similar manner to the NIT. Forexample, a PMT may store information on audio PIDs, and/or video PIDs. APMT stores data on ECM (entitlement control message) PIDs for televisionchannels that are transmitted on a transponder stream, transport stream,or digital broadcast. If, for a first television channel, multipletelevision channels are to be tuned to, the NIT 640 and/or the PMT 657may indicate a second television channel that is to be tuned to when afirst channel is tuned to; this may allow for switching to output of thesecond television channel for different commercials, for example.

A table, such as the NIT, PAT, or PMT may store indications of PIDs thatare related to supplemental audio content for individual channels orother forms of content. For instance, each television channel may beassociated with a different supplemental audio PID. If supplementalaudio content is to be transmitted for a given television channel,packets containing the supplemental audio are transmitted to thereceiver having its associated supplemental audio content PID. This PIDcould be distinct from any of the audio programs of the televisionprogramming (e.g., the first audio program, second audio program (SAP),actual descriptive audio). In some embodiments, supplemental audiocontent is transmitted using the same transponder stream as thetelevision channel or content to which the supplemental audio contentcorresponds. In some embodiments, a single supplemental audio PID isused to identify supplemental audio for multiple television channels,such as all the television channels on a given transponder. A secondidentifier present within the supplemental audio data may be used todistinguish for which television channel the supplemental audio datacorresponds.

Based at least in part on information in the NIT, it may be possible todetermine the proper satellite and transponder to which to tune for aparticular television channel, or to determine the proper antenna andfrequency to which to tune to for a particular television channel. Insome embodiments, the NIT may list a particular frequency to which totune for a particular television channel. Once tuned to the propersatellite/transponder/antenna/frequency, the PMT PID may be used toretrieve a program map table that indicates the PIDs for audio and videostreams of television channels transmitted by that transponder.

While a large portion of storage space of the storage medium 625 may bedevoted to storage of television programming, a portion may be devotedto storage of non-audio/video data, such as the EPG database 630 andother non-video/audio data 631. This “other” data may permit thereceiver 600 to function properly. In some embodiments, at least tengigabytes are allocated to such other data. For example, if the NIT 640is stored by the storage medium 625, it may be part of the othernon-video/audio data 631.

The decoder module 633 may serve to convert encoded video and audio intoa format suitable for output to a display device. For instance, thedecoder module 633 may receive MPEG video and audio from the storagemedium 625, the network interface 620, or the descrambling engine 665 tobe output to a television. MPEG video and audio from the storage medium625 may have been recorded to the DVR database 645 as part of apreviously-recorded television program. The decoder module 633 mayconvert the MPEG video and audio into a format appropriate to bedisplayed by a television or other form of display device and audio intoa format appropriate to be output from speakers, respectively. Thedecoder module 633 may have the ability to convert a finite number oftelevision channel streams received from the storage medium 625, thenetwork interface 620, or the descrambling engine 665 simultaneously.For instance, each of the decoders 634 within the decoder module 633 maybe able to only decode a single television channel at a time.

While a television channel is being decoded by a decoder of the decoders634, the television channel is not necessarily output to a displaydevice via the television interface 635. For instance, a televisionchannel may be decoded but not output to allow for seamless ornear-seamless switching to the television channel when output isdesired. For example, if a second television channel is to be output forpresentation during commercial breaks on a first television channel, thesecond television channel and the first television channel may each becontinuously decoded by different decoders of the decoder module 633 toallow for fast switching between the channels. Without a televisionchannel being continuously decoded, fast switching may be difficult dueat least in part to the format in which the channel is encoded. Forinstance, for MPEG encoding, it may be necessary to first decode anI-frame before certain subsequent (or earlier) received frames may beproperly decoded. The output of the decoder, which is provided to thetelevision interface 635, may be controlled by the control processor610-1, or some other processor. While the decoder module 633 isillustrated as having three decoders 634 (decoder 634-1, decoder 634-2,and decoder 634-3), in other embodiments, a greater or fewer number ofdecoders may be present in the receiver 600.

The television interface 635 may serve to output a signal to atelevision (or another form of display device) in a proper format fordisplay of video and playback of audio. As such, the televisioninterface 635 may output one or more television channels, storedtelevision programming from the storage medium 625 (e.g., televisionprograms from the DVR database 645, television programs from on-demandprogramming 627 and/or information from the EPG database 630) to atelevision for presentation.

Digital Video Recorder (DVR) functionality may permit a televisionchannel to be recorded for a period of time. DVR functionality of thereceiver 600 may be managed by the control processor 610-1. The controlprocessor 610-1 may coordinate the television channel, start time, andstop time of when recording of a television channel is to occur. Invarious embodiments, adaptable content items 176 could be selected andinserted into the presentation stream prior to recording, duringrecording, and/or upon playback of the recording. The DVR database 645may store information related to the recording of television channels.The DVR database 645 may store timers that are used by the controlprocessor 610-1 to determine when a television channel should be tunedto and its programs recorded to the DVR database 645 of storage medium625. In some embodiments, a limited amount of the storage medium 625 maybe devoted to the DVR database 645. Timers may be set by the televisionservice provider and/or one or more users of the receiver 600.

User profiles may also be stored in the storage medium 645 and mayinclude stored user preferences that may be inferred by the televisionreceiver 600 based at least in part on viewing history. The televisionreceiver 600 may communicate user profile information to the servicesystem(s) 102-1, 103-1 to request adaptable content items 176 andcontent items 177 tailored to the inferred user preferences to provisioncomposites 180 in accordance with certain embodiments disclosed herein.The user profiles may include profiles for multiple users or may includea single profile for the television receiver in general. In someembodiments, the user profiles may include preferences for customizedcontent presentation adjustments disclosed herein. The preferences couldinclude customized viewing interests, such as programming content,announcement content, product content, and/or the like that correspondsto augmented content selections relating to recognized actors and/orproducts. The user profiles may further include user feedback, viauser-selectable options, received from the user regardingcustomizations. The feedback data may be used to refine thecustomizations for particular viewers and types of contentcustomizations.

The on-demand programming 627 may represent additional televisionprogramming stored by the storage medium 625. The on-demand programming627 may include television programming that was not recorded to thestorage medium 625 via a timer (either user- or provider-defined).Rather, on-demand programming is programming provided to the receiverdirectly for storage by the receiver and for later presentation to oneor more users. In various embodiments, adaptable content items 176 couldbe selected and inserted into the presentation stream prior to or uponplayback of the recording.

The user interface 650 may include a remote control (physically separatefrom the receiver 600) and/or one or more buttons on the receiver 600that allow a user to interact with the receiver 600. The user interface650 may be used to select a television channel for viewing, viewinformation from the EPG database 630, and/or program a timer stored toDVR database 645, wherein the timer is used to control the DVRfunctionality of the control processor 610-1. The user interface 650 mayalso be used to transmit commands to the receiver 600 and make userselections to customize user preferences, make selections with respectto content items 176, 177, and content composites 180.

Referring back to the tuners 615, television channels received viasatellite (or cable) may contain at least some scrambled data. Packetsof audio and video may be scrambled to prevent unauthorized users (e.g.,nonsubscribers) from receiving television programming without paying thetelevision service provider. When a tuner of the tuners 615 is receivingdata from a particular transponder of a satellite, the transponderstream may be a series of data packets corresponding to multipletelevision channels. Each data packet may contain a packet identifier(PID), which, in combination with the NIT 640 and/or the PMT 657, may bedetermined to be associated with a particular television channel.Particular data packets, referred to as entitlement control messages(ECMs), may be periodically transmitted. ECMs may be associated withanother PID and may be encrypted; the receiver 600 may use thedecryption device 660 to decrypt ECMs. Decryption of an ECM may only bepossible if the user has authorization to access the particulartelevision channel associated with the ECM. When an ECM is determined tocorrespond to a television channel being stored and/or displayed, theECM may be provided to the decryption device 660 for decryption.

The decryption device 660 may be a removable or non-removable smartcard. When the decryption device 660 receives an encrypted ECM, thedecryption device 660 may decrypt the ECM to obtain some number ofcontrol words. In some embodiments, from each ECM received by thedecryption device 660, two control words are obtained. In someembodiments, when the decryption device 660 receives an ECM, it comparesthe ECM to the previously received ECM. If the two ECMs match, thesecond ECM is not decrypted because the same control words would beobtained. In other embodiments, each ECM received by the decryptiondevice 660 is decrypted; however, if a second ECM matches a first ECM,the outputted control words will match; thus, effectively, the secondECM does not affect the control words output by the decryption device660. The decryption device 660 may be permanently part of the receiver600 or may be configured to be inserted and removed from the receiver600. In some embodiments, the control processor 610-1, the tuningmanagement processor 610-2, or other hardware may be able to determinethat a received ECM has not changed from the previous iteration, andtherefore not provide the repeated ECM to the smart card, since the samecontrol word(s) will be obtained.

The tuning management processor 610-2 may be in communication with thetuners 615 and the control processor 610-1. The tuning managementprocessor 610-2 may be configured to receive commands from the controlprocessor 610-1. Such commands may indicate when to start/stop recordinga television channel and/or when to start/stop causing a televisionchannel to be output to a television. The tuning management processor610-2 may control the tuners 615. From the tuners 615, the tuningmanagement processor 610-2 may receive transponder streams of packetizeddata. From the network interface 620, the tuning management processor610-2 may receive network transport stream of packetized data. Aspreviously detailed, some or all of these packets may include a PID thatidentifies the content of the packet.

The tuning management processor 610-2 may be configured to create one ormore PID filters 655 that sort packets received from the tuners 615and/or the network interface 620 based at least in part on the PIDs.When a tuner is initially tuned to a particular frequency (e.g., to aparticular transponder of a satellite), a PID filter may be createdbased at least in part on the PMT data. The PID created, based at leastin part on the PMT data packets, may be known because it is stored aspart of the NIT 640 or another table, such as a program associationtable (PAT). From the PMT data packets, PMT may be constructed by thetuning management processor 610-2.

The PID filters 655 may be configured to filter data packets based atleast in part on PIDs. In some embodiments, the PID filters 655 arecreated and executed by the tuning management processor 610-2. For eachtelevision channel to be output for presentation or recorded, a separatePID filter may be configured. In other embodiments, separate hardwaremay be used to create and execute such PID filters. Depending on atelevision channel selected for recording/viewing, a PID filter may becreated to filter: (1) the video packets associated with the televisionchannel; (2) the audio packets associated with the television channel;and (3), if enabled, supplemental audio content for use in conjunctionwith interactive content. The PMT 657 may store the particularassignments of PIDs for individual television channels. For example, ifa transponder data stream includes multiple television channels, datapackets corresponding to a television channel that is not desired to bestored or displayed by the user may be ignored by the PID filters 655and not routed to the descrambling engine 665, the decryption device660, or the control processor 610-1. As such, only data packetscorresponding to the one or more television channels desired to bestored and/or displayed may be filtered and passed to either thedescrambling engine 665 or the decryption device 660; other data packetsmay be ignored. For each television channel, a stream of video packets,a stream of audio packets (one or both of the audio programs), a streamof supplemental audio content, and/or a stream of ECM packets may bepresent, each stream identified by a PID. In some embodiments, a commonECM stream may be used for multiple television channels. Additional datapackets corresponding to other information, such as updates to the NIT640, may be appropriately routed by the PID filters 655. At a giventime, one or multiple PID filters may be executed by the tuningmanagement processor 610-2.

The descrambling engine 665 may use the control words output by thedecryption device 660 in order to descramble video and/or audiocorresponding to television channels for storage and/or presentation.Video and/or audio data contained in the transponder data streamreceived by the tuners 615 may be scrambled. Video and/or audio data maybe descrambled by the descrambling engine 665 using a particular controlword. Which control word output by the decryption device 660 to be usedfor successful descrambling may be indicated by a scramble controlidentifier present within the data packet containing the scrambled videoor audio. Descrambled video and/or audio may be output by thedescrambling engine 665 to the storage medium 625 for storage (in theDVR database 645) and/or to the decoder module 633 for output to atelevision or other presentation equipment via the television interface635.

For simplicity, the receiver 600 of FIG. 6 has been reduced to a blockdiagram; commonly known parts, such as a power supply, have beenomitted. Further, some routing between the various modules of thereceiver 600 has been illustrated. Such illustrations are for exemplarypurposes only. The state of two modules not being directly or indirectlyconnected does not indicate the modules cannot communicate. Rather,connections between modules of the receiver 600 are intended only toindicate possible common data routing. It should be understood that themodules of the receiver 600 may be combined into a fewer number ofmodules or divided into a greater number of modules.

Further, the components of the television receiver 600 may be part ofanother device, such as built into a television. The television receiver600 may include one or more instances of various computerizedcomponents, such as disclosed in relation to computer systems disclosedfurther herein. While the television receiver 600 has been illustratedas a satellite receiver, it is to be appreciated that techniques belowmay be implemented in other types of television receiving devices, suchas cable receivers, terrestrial receivers, IPTV receivers or the like.In some embodiments, the television receiver 600 may be configured as ahybrid receiving device, capable of receiving content from disparatecommunication networks, such as satellite and terrestrial televisionbroadcasts. In some embodiments, the tuners may be in the form ofnetwork interfaces capable of receiving content from designated networklocations.

FIG. 7 is a block diagram of a system 700 including one non-limitingexample of a computing device 116 configured to make use of, interactwith, and/or at least partially include the subsystem 111, in accordancewith disclosed embodiments of the present disclosure. The computingdevice 116 may be a portable device suitable for sending and receivinginformation to/from the receiver 116 e and over a network to/from remotedata sources (e.g., service providers 103-1 and online content sources112) in accordance with embodiments described herein. For example, invarious embodiments, the computing device 116 may correspond to one ormore of computing devices 116 a, 116 b, 116 c, 116 d depicted in FIG. 2.

In some embodiments, the computing device 116 may be provided with anapplication 751, which may, in some embodiments, correspond to a mobileapplication configured to run on the computing device 116 to facilitatevarious embodiments of this disclosure. For example without limitation,the mobile application 751 may transform the computing device 116 intoan adaptive content splicing device to facilitate features of variousembodiments disclosed herein. In various embodiments, the mobileapplication 751 may allow the device 116 to be configured to provide oneor a combination of the content harvesting engine 236-1, the contentmatching engine 238-1, learning engine 239-1, composite build engine240-1, and/or content splicing engine 242-1 may be provided inconjunction with the content harvesting module 236-2, the contentmatching module 238-2, composite build module 240-2, and/or contentsplicing module 242-2 to implement various functionalities of thecontent composite subsystem 111 into the device 116.

In various embodiments, the application 751 can be any suitable computerprogram that can be installed and run on the computing device 116, and,in some embodiments, the application 751 may not be a mobile app but maybe another type of application, set of applications, and/or otherexecutable code configured to facilitate embodiments disclosed herein.The application 751 may be provided in any suitable way. Fornon-limiting example, the application 751 may be made available from awebsite, an application store, the service provider 102-1, etc. fordownload to the computing device 116; alternatively, it may bepre-installed on the computing device 116.

In various embodiments, the computing device 116 configured with theapplication 751 may provide one or more display screens that may eachinclude one or more user interface elements. A user interface mayinclude any text, image, and/or device that can be displayed on adisplay screen for providing information to a user and/or for receivinguser input. A user interface may include one or more widgets, text, textboxes, text fields, tables, grids, charts, hyperlinks, buttons, lists,combo boxes, checkboxes, radio buttons, and/or the like. As shown inFIG. 7 , the computing device 116 includes a display 720 and inputelements 732 to allow a user to input information into the computingdevice 116. By way of example without limitation, the input elements 732may include one or more of a keypad, a trackball, a touchscreen, atouchpad, a pointing device, a microphone, a voice recognition device,or any other appropriate mechanism for the user to provide input.

In various embodiments, the computing device 116 may pull content items176, content items 177, and/or composites 180 from the receiver 116 eand/or from systems 102-1 and/or 103-1 via the network 120 in order tofacilitate the content composites 180 to a user of the computing device116 through the application 751. The application 751 can include autility that communicates with the receiver 116 e and/or from onlinedata sources via the network 120 to control downloading, displaying,caching, and/or other operations concerning the handling of contentitems 176, content items 177, and/or composites 180. The application 751and the computing device 116 may cooperate with the receiver 116 e tofacilitate tracking of (and customizations of user profiles and otherfeatures disclosed herein based at least in part on) user selections inresponse to content items displayed through the one or more additionalapplications.

The user selection of a user-selectable option corresponding to theapplication 751 may involve any one or combination of various userinputs. The user selection may be in the form of a keyboard/keypadinput, a touch pad input, a track ball input, a mouse input, a voicecommand, etc. For example, the content item may be selected by the userby pointing and clicking on a content item. As another example, acontent item may be selected by an appropriate tap or movement appliedto a touch screen or pad of the computing device 116.

The computing device 116 includes a memory 734 communicatively coupledto a processor 736 (e.g., a microprocessor) for processing the functionsof the computing device 116. The computing device 116 may include atleast one antenna 738 for wireless data transfer to communicate througha cellular network, a wireless provider network, and/or a mobileoperator network, such as GSM, for example without limitation, to sendand receive Short Message Service (SMS) messages or UnstructuredSupplementary Service Data (USSD) messages. The computing device 116 mayalso include a microphone 740 to allow a user to transmit voicecommunication through the computing device 116, and a speaker 742 toallow the user to hear voice communication. The antenna 738 may includea cellular antenna (e.g., for sending and receiving cellular voice anddata communication, such as through a network such as a 3G, 4G, or 5Gnetwork). In addition, the computing device 116 may include one or moreinterfaces in addition to the antenna 738, e.g., a wireless interfacecoupled to an antenna.

The communications interfaces 744 can provide a near field communicationinterface (e.g., contactless interface, Bluetooth, optical interface,infrared interface, etc.) and/or wireless communications interfacescapable of communicating through a cellular network, such as GSM, orthrough Wi-Fi, such as with a wireless local area network (WLAN).Accordingly, the computing device 116 may be capable of transmitting andreceiving information wirelessly through both short range, radiofrequency (RF), cellular, and Wi-Fi connections. The computing device116 may access the network 108 through a wireless link to an accesspoint. For example, a computing device 116 may access the network 108through one or more access points 706. The access points 706 may be ofany suitable type or types. For example, an access point 706 may be acellular base station, an access point for wireless local area network(e.g., a Wi-Fi access point), an access point for wireless personal areanetwork (e.g., a Bluetooth access point), etc. The access point 706 mayconnect the computing device 116 to the network 108, which may includethe Internet, an intranet, a local area network, private communicationnetworks, etc. In some embodiments, the communications interfaces 744may allow computing device 116 to receive programming content cast fromthe television receiver. For example, the programming content from thetelevision receiver may be indirectly transmitted via a local network(e.g., via Wi-Fi) or directly transmitted to the computing device via acasting device integrated with the television receiver or coupled to thetelevision receiver (e.g., via a dongle). As another example, thetelevision receiver may cast programming content to the computing devicevia a wired connection (e.g., via one or more of HDMI, USB, lightningconnector, etc.). Some embodiments may provide for simulcasting suchthat the same programming that is being displayed on the display deviceis being displayed on the computing device 116 simultaneously orsubstantially simultaneously.

The computing device 116 can also include at least one computer-readablemedium 746 coupled to the processor 736, which stores applicationprograms and other computer code instructions for operating the device,such as an operating system (OS) 748. In some embodiments, theapplication 751 may be stored in the memory 734 and/or computer-readablemedia 746. Again, the example of computing device 116 is non-limiting.Other devices, such as those disclosed herein, may be used.

The mobile computing device 736 can additionally include an integratedcamera 754, capable of capturing images and/or video, and outputelements 752. In certain embodiments, the mobile computing device 736may include a non-transitory computer-readable storage medium, e.g.,memory 734, for storing images captured with the camera 754. Inalternative embodiments, the mobile computing device 736 receives imagedata from an image capture device that is not integrated with the mobilecomputing device 736 (e.g., from the receiver 116 e and/or the onlinecontent sources 112).

In some embodiments, the computer-readable medium 746 can also includean image-processing engine 746(a). The image-processing engine 746(a)can perform image processing of images captured by the camera 754 toperform recognition of programming content 202 in accordance withembodiments disclosed herein. The image-processing engine 746(a) mayanalyze the images and generate accessible interface elements using theidentified and/or analyzed images. With some embodiments, theimage-processing engine 746(a) is separate from the application 751. Inother embodiments, the application 751 may include the image-processingengine 746(a). In some embodiments, the computing device 116 providesthe captured images to the receiver 116 e, and the receiver 116 eperforms the image processing and analysis. In some embodiments, thecomputing device 116 provides the captured images to the serviceprovider 102-1, and the service provider performs the image processingand analysis.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware might also be used, and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

A computer system as illustrated in FIG. 8 may be incorporated as partof the previously described computerized devices, such as a televisionreceiver 116 e, device(s) 116, and/or content provider system 102-1.FIG. 8 provides a schematic illustration of one embodiment of a computersystem 800 that can perform various steps of the methods provided byvarious embodiments. It should be noted that FIG. 8 is meant only toprovide a generalized illustration of various components, any or all ofwhich may be utilized as appropriate. FIG. 8 , therefore, broadlyillustrates how individual system elements may be implemented in arelatively separated or relatively more integrated manner.

The computer system 800 is shown comprising hardware elements that canbe electrically coupled via a bus 805 (or may otherwise be incommunication, as appropriate). The hardware elements may include one ormore processors 810, including without limitation one or moregeneral-purpose processors and/or one or more special-purpose processors(such as digital signal processing chips, graphics accelerationprocessors, video decoders, and/or the like); one or more input devices815, which can include without limitation a mouse, a keyboard, remotecontrol, and/or the like; and one or more output devices 820, which caninclude without limitation a display device, a printer, and/or the like.

The computer system 800 may further include (and/or be in communicationwith) one or more non-transitory storage devices 825, which cancomprise, without limitation, local and/or network accessible storage,and/or can include, without limitation, a disk drive, a drive array, anoptical storage device, a solid-state storage device, such as a randomaccess memory (“RAM”), and/or a read-only memory (“ROM”), which can beprogrammable, flash-updateable and/or the like. Such storage devices maybe configured to implement any appropriate data stores, includingwithout limitation, various file systems, database structures, and/orthe like.

The computer system 800 might also include a communications subsystem830, which can include without limitation a modem, a network card(wireless or wired), an infrared communication device, a wirelesscommunication device, and/or a chipset (such as a Bluetooth™ device, an802.11 device, a Wi-Fi device, a WiMAX device, cellular communicationdevice, etc.), and/or the like. The communications subsystem 830 maypermit data to be exchanged with a network (such as the networkdescribed below, to name one example), other computer systems, and/orany other devices described herein. In many embodiments, the computersystem 800 will further comprise a working memory 835, which can includea RAM or ROM device, as described above.

The computer system 800 also can comprise software elements, shown asbeing currently located within the working memory 835, including anoperating system 840, device drivers, executable libraries, and/or othercode, such as one or more application programs 845, which may comprisecomputer programs provided by various embodiments, and/or may bedesigned to implement methods, and/or configure systems, provided byother embodiments, as described herein. Merely by way of example, one ormore procedures described with respect to the method(s) discussed abovemight be implemented as code and/or instructions executable by acomputer (and/or a processor within a computer); in an aspect, then,such code and/or instructions can be used to configure and/or adapt ageneral purpose computer (or other device) to perform one or moreoperations in accordance with the described methods.

A set of these instructions and/or code might be stored on anon-transitory computer-readable storage medium, such as thenon-transitory storage device(s) 825 described above. In some cases, thestorage medium might be incorporated within a computer system, such ascomputer system 800. In other embodiments, the storage medium might beseparate from a computer system (e.g., a removable medium, such as acompact disc), and/or provided in an installation package, such that thestorage medium can be used to program, configure, and/or adapt a generalpurpose computer with the instructions/code stored thereon. Theseinstructions might take the form of executable code, which is executableby the computer system 800 and/or might take the form of source and/orinstallable code, which, upon compilation and/or installation on thecomputer system 800 (e.g., using any of a variety of generally availablecompilers, installation programs, compression/decompression utilities,etc.), then takes the form of executable code.

As mentioned above, in one aspect, some embodiments may employ acomputer system (such as the computer system 800) to perform methods inaccordance with various embodiments of the invention. According to a setof embodiments, some or all of the procedures of such methods areperformed by the computer system 800 in response to processor 810executing one or more sequences of one or more instructions (which mightbe incorporated into the operating system 840 and/or other code, such asan application program 845) contained in the working memory 835. Suchinstructions may be read into the working memory 835 from anothercomputer-readable medium, such as one or more of the non-transitorystorage device(s) 825. Merely by way of example, execution of thesequences of instructions contained in the working memory 835 mightcause the processor(s) 810 to perform one or more procedures of themethods described herein.

The terms “machine-readable medium,” “machine-readable media,”“computer-readable storage medium,” “computer-readable storage media,”“computer-readable medium,” “computer-readable media,”“processor-readable medium,” “processor-readable media,” and/or liketerms as used herein, refer to any medium that participates in providingdata that causes a machine to operate in a specific fashion. Thesemediums may be non-transitory. In an embodiment implemented using thecomputer system 800, various computer-readable media might be involvedin providing instructions/code to processor(s) 810 for execution and/ormight be used to store and/or carry such instructions/code. In manyimplementations, a computer-readable medium is a physical and/ortangible storage medium. Such a medium may take the form of anon-volatile media or volatile media. Non-volatile media include, forexample, optical and/or magnetic disks, such as the non-transitorystorage device(s) 825. Volatile media include, without limitation,dynamic memory, such as the working memory 835.

Common forms of physical and/or tangible computer-readable mediainclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, or any other magnetic medium, a CD-ROM, any other opticalmedium, any other physical medium with patterns of marks, a RAM, a PROM,EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any othermedium from which a computer can read instructions and/or code.

Various forms of computer-readable media may be involved in carrying oneor more sequences of one or more instructions to the processor(s) 810for execution. Merely by way of example, the instructions may initiallybe carried on a magnetic disk and/or optical disc of a remote computer.A remote computer might load the instructions into its dynamic memoryand send the instructions as signals over a transmission medium to bereceived and/or executed by the computer system 800.

The communications subsystem 830 (and/or components thereof) generallywill receive signals, and the bus 805 then might carry the signals(and/or the data, instructions, etc. carried by the signals) to theworking memory 835, from which the processor(s) 810 retrieves andexecutes the instructions. The instructions received by the workingmemory 835 may optionally be stored on a non-transitory storage device825 either before or after execution by the processor(s) 810.

It should further be understood that the components of computer system800 can be distributed across a network. For example, some processingmay be performed in one location using a first processor while otherprocessing may be performed by another processor remote from the firstprocessor. Other components of computer system 800 may be similarlydistributed. As such, computer system 800 may be interpreted as adistributed computing system that performs processing in multiplelocations. In some instances, computer system 800 may be interpreted asa single computing device, such as a distinct laptop, desktop computer,or the like, depending on the context.

The methods, systems, and devices discussed above are examples. Variousconfigurations may omit, substitute, or add various procedures orcomponents as appropriate. For instance, in alternative configurations,the methods may be performed in an order different from that described,and/or various stages may be added, omitted, and/or combined. Also,features described with respect to certain configurations may becombined in various other configurations. Different aspects and elementsof the configurations may be combined in a similar manner. Also,technology evolves and, thus, many of the elements are examples and donot limit the scope of the disclosure or claims.

Specific details are given in the description to provide a thoroughunderstanding of example configurations (including implementations).However, configurations may be practiced without these specific details.For example, well-known circuits, processes, algorithms, structures, andtechniques have been shown without unnecessary detail in order to avoidobscuring the configurations. This description provides exampleconfigurations only, and does not limit the scope, applicability, orconfigurations of the claims. Rather, the preceding description of theconfigurations will provide those skilled in the art with an enablingdescription for implementing described techniques. Various changes maybe made in the function and arrangement of elements without departingfrom the spirit or scope of the disclosure.

Also, configurations may be described as a process which is depicted asa flow diagram or block diagram. Although each may describe theoperations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be rearranged. A process may have additional steps notincluded in the figure. Furthermore, examples of the methods may beimplemented by hardware, software, firmware, middleware, microcode,hardware description languages, or any combination thereof. Whenimplemented in software, firmware, middleware, or microcode, the programcode or code segments to perform the necessary tasks may be stored in anon-transitory computer-readable medium such as a storage medium.Processors may perform the described tasks.

Having described several example configurations, various modifications,alternative constructions, and equivalents may be used without departingfrom the spirit of the disclosure. For example, the above elements maybe components of a larger system, wherein other rules may takeprecedence over or otherwise modify the application of the invention.Also, a number of steps may be undertaken before, during, or after theabove elements are considered.

Furthermore, the example embodiments described herein may be implementedas logical operations in a computing device in a networked computingsystem environment. The logical operations may be implemented as: (i) asequence of computer implemented instructions, steps, or program modulesrunning on a computing device; and (ii) interconnected logic or hardwaremodules running within a computing device.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

Also, the terms in the claims have their plain, ordinary meaning unlessotherwise explicitly and clearly defined by the patentee. The indefinitearticles “a” or “an,” as used in the claims, are defined herein to meanone or more than one of the element that the particular articleintroduces; and subsequent use of the definite article “the” is notintended to negate that meaning. Furthermore, the use of ordinal numberterms, such as “first,” “second,” etc., to clarify different elements inthe claims is not intended to impart a particular position in a series,or any other sequential character or order, to the elements to which theordinal number terms have been applied.

What is claimed:
 1. A method comprising: transmitting, by one or moreprocessing devices, transmission of a first content composite thatcomprises a set of one or more audio and/or video packets correspondingto audio and/or video content, where, consequent to delivery of thefirst content composite to a first endpoint media device or a secondendpoint media device, the first endpoint media device or the secondendpoint media device performs at least one operation relating to acontent item of the first content composite represented by one or moreimages; aggregating a set of observation data corresponding toindications of detected operations associated with the first endpointmedia device and/or the second endpoint media device and mapped to a setof one or more content items, the set of one or more content itemscomprising the content item of the first content composite; and based atleast in part on the set of observation data, adapting a second contentcomposite so that the second content composite is modified based atleast in part on a second content item, and providing the second contentcomposite to facilitate presentation of the second content composite byone or both of the first endpoint media device and the second endpointmedia device.
 2. The method as recited in claim 1, further comprising:processing at least one communication received from the first endpointmedia device, the second endpoint media device, or a remote system thatis remote from the one or more processing devices, the first endpointmedia device, and the second endpoint media device, the at least onecommunication indicative of the at least one operation relating to thecontent item of the first content composite.
 3. The method as recited inclaim 2, where the aggregating the set of observation data is based atleast in part on the processing the at least one communication.
 4. Themethod as recited in claim 1, further comprising: creating the firstcontent composite for delivery in a packet stream from a contentprovider system via one or more networks, where the first contentcomposite comprises an adaptable content item corresponding to the setof one or more audio and/or video packets.
 5. The method as recited inclaim 1, where, consequent to a selection of a user-selectable interfaceelement mapped to the second content item, a third content item istransferred to one or both of the first endpoint media device and thesecond endpoint media device.
 6. The method as recited in claim 1, wherethe at least one operation relating to the second content itemcorresponds to an image capture of at least a portion of an imagefacilitated by the second content item.
 7. The method as recited inclaim 1, further comprising: based at least in part on the aggregatedset of observation data, learning, by the one or more processingdevices, one or more metrics of media device interactions with a set ofone or more content composites and mapping the one or more metrics tothe first endpoint media device and/or the second endpoint media device;and where the adapting the second content composite is based at least inpart on the learned one or more metrics of media device interactions. 8.A system comprising: one or more processing devices; and memorycommunicatively coupled with and readable by the one or more processingdevices and having stored therein processor-readable instructions which,when executed by the one or more processing devices, cause the one ormore processing devices to perform: transmitting of a first contentcomposite that comprises a set of one or more audio and/or video packetscorresponding to audio and/or video content, where, consequent todelivery of the first content composite to a first endpoint media deviceor a second endpoint media device, the first endpoint media device orthe second endpoint media device performs at least one operationrelating to a content item of the first content composite represented byone or more images; aggregating a set of observation data correspondingto indications of detected operations associated with the first endpointmedia device and/or the second endpoint media device and mapped to a setof one or more content items, the set of one or more content itemscomprising the content item of the first content composite; and based atleast in part on the set of observation data, adapting a second contentcomposite so that the second content composite is modified based atleast in part on a second content item, and providing the second contentcomposite to facilitate presentation of the second content composite byone or both of the first endpoint media device and the second endpointmedia device.
 9. The system as recited in claim 8, the one or moreprocessing devices further to perform: processing at least onecommunication received from the first endpoint media device, the secondendpoint media device, or a remote system that is remote from the one ormore processing devices, the first endpoint media device, and the secondendpoint media device, the at least one communication indicative of theat least one operation relating to the content item of the first contentcomposite.
 10. The system as recited in claim 9, where the aggregatingthe set of observation data is based at least in part on the processingthe at least one communication.
 11. The system as recited in claim 8,the one or more processing devices further to perform: creating thefirst content composite for delivery in a packet stream from a contentprovider system via one or more networks, where the first contentcomposite comprises an adaptable content item corresponding to the setof one or more audio and/or video packets.
 12. The system as recited inclaim 8, where, consequent to a selection of a user-selectable interfaceelement mapped to the second content item, a third content item istransferred to one or both of the first endpoint media device and thesecond endpoint media device.
 13. The system as recited in claim 8,where the at least one operation relating to the second content itemcorresponds to an image capture of at least a portion of an imagefacilitated by the second content item.
 14. The system as recited inclaim 8, the one or more processing devices further to perform: based atleast in part on the aggregated set of observation data, learning one ormore metrics of media device interactions with a set of one or morecontent composites and mapping the one or more metrics to the firstendpoint media device and/or the second endpoint media device; and wherethe adapting the second content composite is based at least in part onthe learned one or more metrics of media device interactions.
 15. One ormore non-transitory, machine-readable media having machine-readableinstructions thereon which, when executed by one or more processingdevices, cause the one or more processing devices to perform:transmitting of a first content composite that comprises a set of one ormore audio and/or video packets corresponding to audio and/or videocontent, where, consequent to delivery of the first content composite toa first endpoint media device or a second endpoint media device, thefirst endpoint media device or the second endpoint media device performsat least one operation relating to a content item of the first contentcomposite represented by one or more images; aggregating a set ofobservation data corresponding to indications of detected operationsassociated with the first endpoint media device and/or the secondendpoint media device and mapped to a set of one or more content items,the set of one or more content items comprising the content item of thefirst content composite; and based at least in part on the set ofobservation data, adapting a second content composite so that the secondcontent composite is modified based at least in part on a second contentitem, and providing the second content composite to facilitatepresentation of the second content composite by one or both of the firstendpoint media device and the second endpoint media device.
 16. The oneor more non-transitory, machine-readable media as recited in claim 15,the one or more processing devices further to perform: processing atleast one communication received from the first endpoint media device,the second endpoint media device, or a remote system that is remote fromthe one or more processing devices, the first endpoint media device, andthe second endpoint media device, the at least one communicationindicative of the at least one operation relating to the content item ofthe first content composite.
 17. The one or more non-transitory,machine-readable media as recited in claim 16, where the aggregating theset of observation data is based at least in part on the processing theat least one communication.
 18. The one or more non-transitory,machine-readable media as recited in claim 15, the one or moreprocessing devices further to perform: creating the first contentcomposite for delivery in a packet stream from a content provider systemvia one or more networks, where the first content composite comprises anadaptable content item corresponding to the set of one or more audioand/or video packets.
 19. The one or more non-transitory,machine-readable media as recited in claim 15, where, consequent to aselection of a user-selectable interface element mapped to the secondcontent item, a third content item is transferred to one or both of thefirst endpoint media device and the second endpoint media device. 20.The one or more non-transitory, machine-readable media as recited inclaim 15, where the at least one operation relating to the secondcontent item corresponds to an image capture of at least a portion of animage facilitated by the second content item.